Design and Development of a Small Man- Portable Fish-Finding Autonomous Underwater Vehicle (AUV)

Underwater inspections are a challenging research field. Inspections are traditionally made using acoustic sensing from surface vessels or ROVs (remotely operated vehicles) such as the “Sea Wolf” from ATLAS Hydrographic GmbH [1]. In recent years the trend of underwater operations has shifted towards using AUVs. Such vehicles are fitted with payload and sensing suite to perform high accuracy navigation and sensing using an INS (inertial navigation system) and DVL (Doppler velocity log). Payloads can be integrated onto the AUV platform to support activities such as sub-bottom profiling, seabed topography mapping, high-resolution acoustic imaging and video imaging. Some advantages of deploying AUVs rather than ROVs or surface vessels are enhanced data quality, operational depth and range, and durability even in hazardous environments. In fishing operations, fishermen can detect pelagic fish in the water column by use of the on-board echo sounder on the fishing vessel. This is wellknown and used in most fishing activities. The echo sounder does however lack to produce any information or parameterization of the distribution of fish species present, estimated bycatch and discard rates.

By-catch and discards refers to the portion of a catch of fish which is not retained onboard during fishing operations and is returned to the sea, often dead or dying. To maintain a sustainable aquaculture and force commercial fishing to strive for a cleaner catch, a new set of regulations have been introduced to the industry in 2015. The “Discarding and the landing obligation” [2] presents two options for commercial fisheries regarding bycatch and discards. Either fishermen must use their limited quota for by-catch causing them to abandon all fishing activities when used up, or they must invest time and money towards cleaner fishing effectively reducing the amount of by-catch and discards. In EU waters, the discard and by-catch rates are estimated to be in the range of 15% to 40% in recent EU reports [3]. Worldwide estimations quantify to as much as 20 million tons of marine life annually.

To support such operations, the aforementioned AUVs are rendered too costly (several million e), complex to operate and the physical size of these are rendering them difficult to both deploy, recover and store. Therefore, a demand for smaller, affordable, less complex and possibly expendable AUVs arises. ATLAS MARIDAN are leveraging more than 20 years of experience in the AUV field along with support from the Technical University of Denmark to research and develop such a vehicle with absolute minimum requirements to hardware and sensors along with hard constraints on size and production cost.

Organization of this paper

This paper presents the current version of a fish-finding AUV concept as an end-to-end design and development product. In Section II, a brief overview of the AUV concept and architecture is presented. In Section III, the realization of the fish-finding AUV is presented. In Section IV is the on-board intelligence w.r.t. detecting, locating and classifying fish and fish species presented. In Section V is the mission control interface, data visualization interface and sea trial results presented. Finally, a conclusion is found in Section VI.

Novelty in this work

During this research, the primary focus has been to design and develop a man-portable, low-cost and commercially available complete AUV system. This work proposes a novel concept of operating the AUV as assistance or aid in commercial fisheries to reduce the effective amount of by-catch and discards in EU waters.

Other than the mechanical design of the AUV, the focus has been on minimizing the need for expensive hardware and sensors and in turn maximizing the yield of the integrated low-cost parts. The system furthermore presents a state-of-the-art camera system based on AI to efficiently detect, locate and classify fish and fish species even in poor conditions. The need for an extensive surface setup for operating the AUV has been re-evaluated and re-designed such that the entire system is operated from an application effectively available for any tablet, mobile phone and PC. Finally, all of the fish-finding AUV versions have been designed, built, tested and qualified at sea.

A sketch of the proposed concept is illustrated in Figure 1. Here a fisherman has detected a school of fish using his on-board echo sounder. No information of this specific school of fish other than echo returns and guesswork based on experience is available. Not knowing expected by-catch rates or possible discards, the proposed AUV is deployed. As shown in the sketch, the AUV is to descent into the water column where the fish are detected and move towards/ circulate the school of fish while gathering and processing data from on-board cameras and processors. When the AUV emerges, the results are streamed to the fisherman. This effectively allows the fisherman to make facts based on-site decisions to substantially minimize the risk of by-catch by making sure necessary data and information of that specific school of fish are made available to him. Directly correlated with by-catch reduction is maximization of exploitation of quotas as well as maximizing the yield of the main catch.

Figure 1:Sketch of concept for the proposed fishfinding AUV.

Table 1:Overall design and performance requirements.

Figure 2:CAD rendering of previous and current version of the AUV. (a) Mk. 1.0, (b) Mk. 1.1, (c) Mk. 1.2.

Figure 3:An overview of the main components of the AUV.

Figure 4:Photograph of the fish-finding AUV.

Table 2:Estimated power budget for the fish-finding AUV.

Figure 5:Experimental results from evaluating OFDNet with data obtained of herring in the Baltic Sea off the coast of Rugen in Germany (a) and (b), and data obtained of other species in the Weser in Bremen, Germany (c).

Figure 6:Mission control interface.

Figure 7:Data offloading, visualization and results review application.

Figure 8:Selected AUV system log data from a sea trial for a fish-finding mission.

Figure 9:Selected AUV system log data from a sea trial for an alternative operational mode of altitude keeping in heading control.

To realize a vehicle capable of carrying out missions as sketched in Figure 1, several aspects in AUV electrical and mechanical design needs to be examined carefully for the vehicle to succeed in both functionality and requirements to size and cost. Some considerations are mentioned in the following: A minimalist singletube mechanical design with constraints on size and weight is needed. This must include a propulsion system enabling forward, backward, pitch and yaw movement. To perform the basics of navigation, the design must include navigational sensors as IMU (inertial measurement unit), depth sensor and GPS. Interfacing sensors and actively controlling the AUV must be done on-board using embedded processors. For video imaging, low-light cameras and external lights are needed to acquire data even at depths where no natural illumination is available. Communication is needed to transfer and execute missions on the AUV, to manually control it or to retrieve the results from a completed mission. A summary of overall design and performance parameters for the AUV is shown in Table 1.

In this section the mechanical, electrical and system design of the fish-finding AUV is presented.

Design

The aim and vision for the first-generation vehicles is to design and build a cost-effective AUV based on “off-the-shelf” components. This keeps price and production time to a minimum while a very capable AUV remains delivered. To keep the time between idea and functional prototypes low, rapid prototyping into is adapted into the workflow. Techniques such as additive manufacturing allows a continuous focus on function and form rather than production specific issues. This is evident in both the nose and rear section of iterations in the first-generation vehicles as can be seen on Figure 2. Therefore, mentioned sections are in-house designed and prototyped using laser sintered polyamide. The design of the current version of the fish-finding AUV is modular to further assist ongoing research and development activities. Incremental upgrades can be achieved by simply changing the sensor head, pressure tube or propulsion section. For the propulsion system, a simple and rugged quad-thruster design has been chosen. This leaves the least amount of moving parts and eliminates the complexity of adding additional servos to achieve pitch and yaw motion. The hull of the AUV is designed around a single pressure tube in acrylic material. The vehicle fits an antenna on top which carries a GPS receiver for location-fix when positioned in the surface along with a Wi-Fi antenna for communication and streaming of data. Furthermore, the antenna holds a recovery light along with traditionally colored navigational lights (red/green) known from sailing boats.

Other than the quad-thruster arrangement, the aft section holds an interface for turning on/off the vehicle and charging the batteries. The battery power supply is composed by two five-cell LiPo batteries in parallel with a total capacity of 10000mAh. The mid-section fits two carrying handles on top, and in the frontend of the mid-section are two slightly angled wings mounted for underwater stability and to provide a downward force when moving forward. The sensor head located in the bow section carries a forward facing and port-side facing camera. Both cameras are supported by external underwater light. In addition to requirements from Table 1, an altimeter is placed on the bottom of the AUV. This supply both altitude and water-column data and can potentially support alternative operational modes other than aiding in fishing operations. The current design of the vehicle concludes in a weight of 9:8kg, a length of 1m and a tube diameter of 0:1m. Given the low weight, small size and design, the AUV is man-portable and be easily launched and recovered from both a boat and shoreline.

System control interface

For control and interfacing the vehicle, currently a Beagle-Bone Blue [4] is utilized. This is a Linux-based embedded processor designed for robotics with a wide range of support for connectivity and sensors. Furthermore, the Beagle Bone Blue already integrates several sensors, here among a 9-axis IMU. A Raspberry Pi 3 Model B [5] is currently utilized in addition to the Beagle Bone. This handles interfacing with cameras and performs the processing of the data acquired by these. As the processing algorithm is based on current state-of-the-art deep learning models, the Intel Movidius NCS (Neural Control Stick) [6] is used by the Raspberry Pi to execute and accelerate the deep learning model. The backbone software for control of the AUV, communication and logging are executed from the Beagle Bone Blue. This software is MARIDAN’s AUV Core software which has been well proven and used for more than 15 years in both AUVs and ROVs. Furthermore, this also include the modularity of backseat drivers to effectively allow external processes to gain control of vehicle while planned missions are being interrupted [7]. Such processes could be: Actively fish school tracking at longer ranges using multi-beam sonars. Signals from video imaging processing. More difficult tasks for underwater objects inspection and tracking requiring multi-sensor processing as explained in [8,9].

Components and power budget

The main components of the AUV system are shown in Figure 3. From this it can be seen that the motors, ESCs, underwater lights, pressure sensor, leak sensor and USB low-light cameras are supplied by Blue Robotics [10] and all are off-the-shelf components. The altimeter is supplied by Marine Sonic Technology [11] and the GPS is from U-blox [12]. An estimated power budget of the entire system is given in Table 2. From here it can be seen that the estimated power consumption for the vehicle IDLE state is 9W and for an average mission state it is 75:1 W. If the assumed average voltage of the battery is 19:5V, the estimated duration of the AUV is t=Q=A. For IDLE state this is 21h and 40min. For the average mission state, the estimated operational time is 2h and 36min. An actual photo of the current version of the vehicle is shown in Figure 4.

To enable the AUV to estimate by-catch and discard rates of a school of fish, the Optical Fish Detection Network (OFD Net) [13] is introduced. This is a deep learning model that carries out the task of fish detection, localization and species classification in data obtained by cameras. OFD Net is based on a current state-of-the-art object detection architecture known as SSD (Single Shot Multi box Detector) [14] using a Mobile Net feature extractor [15]. A database of fish used for training have been collected to support operations in the poorly conditioned North and Baltic Sea. The training of the deep learning model has been carried out on machines with access to NVIDIA GPUs and as previously described the trained model is deployed on a Raspberry Pi with an Intel Movidius NCS attached. By executing the deep learning related routines on the Intel Movidius NCS an increase of effective framerate of more than 1000% is shown. This is compared to a vanilla Raspberry Pi system. For testing the on-board intelligence, a dataset in previously unexplored conditions is collected. This dataset contains 97 fish distributed among two out of three initial considered fish species classes; herring, mackerel and “other species”. By evaluating this dataset shows that OFD Net successfully detects 66:7 % of the fish included in the dataset and furthermore classifies 89:7 % of these correctly. Figure 5 shows three images from the obtained dataset evaluated by OFD Net. Here harsh conditions, low visibility and motion blur are clearly identified and represented; however, it is shown that OFD Net remains capable to successfully detect and classify fish correctly for most of these samples.

In this section the mission control interface and mission results visualization application are presented. This composes the required surface setup to operate the AUV. Finally, a presentation of two typical systems logs obtained by testing the complete system on sea trials is given.

Control system

As the small AUV system adopts the long-term proven AUV Core software, it heavily benefits and relies on experience made through extensive work with larger surveying platforms by MARIDAN. The controllers of the AUV rely on simple cascaded PID control loops which are modalized and extendable through AUV Core. AUV Core furthermore presents a highly flexible and extensible mission description and control language. For the fish-finding application a mostly fixed mission pattern is adopted. This remains however to be described by the aforementioned language, designed in blocks that translates to state machines and with parameter settings configurable from the surface setup. The entire workflow for successfully operating the AUV for fish-finding purpose can be described as follows: Deploy the AUV by hand. Set a lead in track to achieve a desired depth. Adopt a circling pattern to find, obtain and process data of fish. When the AUV emerges, the AUV goes to recovery point and keeps position. The AUV transmits high priority information to the mother vessel/surface setup when connection is established. Recover the AUV and download mission and/or run next mission. Using AUV Core, other mission scenarios to gather different data can easily be generated. The vehicle readily includes a seabed following mission to record data at a fixed altitude.

Surface setup

The only hardware needed for operating the AUV is simply a device capable of running the developed GUI (graphical user interface) applications. The designed software can be cross compiled to support both Linux, Windows, Mac OS, Android and iOS. For operating this software, and the AUV, currently a Samsung Galaxy tablet utilized. The application shown in Fig. 6 makes up the mission control interface. The GUI furthermore generates the mission description, uploads and starts missions, displays basic vehicle information and status and is finally extensible, modifiable and adaptable based on the AUV Core mission control language.

When the tablet is tilted to horizontal view, the mission control application in Figure 6 reveals a joypad like interface available for manually steering the AUV. Another section in the mission control application allows the user to generate seabed following missions. This controls the vehicle to maintain a pre-configured altitude and heading. Possible applications for such missions could include seabed video imaging when pointing cameras downwards or possibly sonar-based surveys with less requirements to navigation accuracy.

Figure 7 shows the data offloading, visualization and mission results review application. This application connects to the AUV when emerging after a completed mission and downloads the immediate and high priority information such as expected by-catch rates, amount of fish detected and their depth position in the water column. Furthermore, it downloads the image files along with detection results for on-site user verification and inspection.

All versions of the AUV have been thoroughly qualified and tested at MARIDAN’s test facilities in Oresund. Further-more, the current vehicle has attended several sea-trials in both Oresund, the North and Baltic Sea. Here it has been launched and recovered from both the shore as well as from both small and large vessels. The current AUV have attended multiple surveys for both fishing operations and seabed video imaging.

Figure 8 & 9 show typical results from two logged missions obtained on sea trials during the spring of 2018. In Figure 8 a circling diving pattern for the fish-finding concept is completed. Here it can be seen that the depth of the AUV linearly increases and the heading indicates the circling behavior using a fixed time per circle. In Figure 9 a seabed following diving pattern is completed. Here it can be seen that the AUV uses the altimeter as reference to dive towards an altitude of 1m and maintains this altitude while maintaining the reference heading.

This paper has presented the current results from the research and work in designing and developing a small man portable AUV. The current version of this AUV realizes the conceptual use-case of supporting commercial fisheries. The current version of the AUV obtains a depth rating of 30m, a total mass in air of 9:8kg and an estimated mission durability of 2h and 36min. The AUV features on-board AI to perform real-time fish detection, localization and species classification in real-time during AUV missions. This is based on images obtained from the on-board cameras. The AUV system leverages the extensive and quality proven AUV Core software along with software tools to operate the entire system from a tablet device or similar. The AUV have been successfully tested on multiple sea-trials in different weather states, locations and launch/recovery procedures and the processing algorithms have been qualified using data obtained in real environments in EU waters.

This work was supported by oXeanpedia (www. oxeanpedia. de), a ThyssenKrupp Garage start-up and by ATLAS MARIDAN ApS. Thanks to all colleagues from both oXeanpedia and ATLAS MARIDAN, in particular Marc Schiemann, marc.schiemann@ oxeanpedia.de and Claus Eriksen, cer@altasmaridan.com, for your organizational support and contributions to the work.

1. Konnecke S (2013) ATLAS electronic group: Atlas hydrographic, Bremen, Germany.
2. (2013) Discarding and the landing obligation, European Commission, Europe.
3. (2011) Impact assessment of discard reducing policies. EU Discard Annex, Europe.
4. https://beagleboard.org/blue
5. https://www.raspberrypi.org/
6. https://github.com/movidius/ncsdk
7. Jacobi M, Karimanzira D (2015) Guidance of AUVs for autonomous underwater inspection. Special Issue: Maritime Systems /Jrgen Beyerer- automation technology 63(5): 380-388.
8. Marco J, Divas K (2014) Multi sensor underwater pipeline tracking with AUVs. Oceans-St. John’s, pp. 1-6.
9. Marco J, Divas K (2013) Underwater pipeline and cable inspection using autonomous underwater vehicles. MTS/IEEE OCEANS-Bergen, pp. 1-6.
10. https://www.bluerobotics.com/
11. https://www.marinesonic.com/
12. https://www.u-blox.com/en
13. Christensen JH, Mogensen LV, Andersen JC, Galeazzi R (2018) Detection, localization and classification of fish and fish species in poor conditions using convolutional neural networks. IEEE OES Autonomous Underwater Vehicle Symposium 18, Portugal.
14. Liu W, Anguelov D, Erhan D, Szegedy C, Reed SE et al. (2015) SSD: Single shot multibox detector. CoRR, vol. abs/1512.02325, USA.
15. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W et al. (2017) Mobile nets: Efficient convolutional neural networks for mobile vision applications, CoRR, vol. abs/1704. 04861, USA.