Prediction of fish mortality based on a probabilistic anomaly detection approach for recirculating aquaculture system facilities
13
Citation
59
Reference
10
Related Paper
Citation Trend
Abstract:
Aquaculture is a fundamental sector of the food industry nowadays. However, to become a sustainable and more profitable industry, it is necessary to monitor several associated parameters, such as temperature, salinity, ammonia, potential of hydrogen, nitrogen dioxide, bromine, among others. Their regular and simultaneous monitoring is expected to predict and avoid catastrophes, such as abnormal fish mortality rates. In this paper, we propose a novel anomaly detection approach for the early prediction of high fish mortality based on a multivariate Gaussian probability model. The goal of this approach is to determine the correlation between the number of daily registered physicochemical parameters of the fish tank water and the fish mortality. The proposed machine learning model was fitted with data from the weaning and pre-fattening phases of Senegalese sole (Solea senegalensis) collected over 2018, 2019, and 2020. This approach is suitable for real-time tracking and successful prediction of up to 80% of the high fish mortality rates. To the best of our knowledge, the proposed anomaly detection approach is the first time studied and applied in the framework of the aquaculture industry.Keywords:
Anomaly (physics)
Anomaly (physics)
Benchmark (surveying)
Streaming Data
Cite
Citations (4)
An unresolved challenge for all kind of temporal data is the reliable anomaly detection, especially when adaptability is required in the case of non-stationary time series or when the nature of future anomalies is unknown or only vaguely defined. Most of the current anomaly detection algorithms follow the general idea to classify an anomaly as a significant deviation from the prediction. In this paper we present a comparative study where several online anomaly detection algorithms are compared on the large Yahoo Webscope S5 anomaly benchmark. We show that a relatively Simple Online Regression Anomaly Detector (SORAD) is quite successful compared to other anomaly detectors. We discuss the importance of several adaptive and online elements of the algorithm and their influence on the overall anomaly detection accuracy.
Anomaly (physics)
Benchmark (surveying)
Adaptability
Cite
Citations (33)
In this paper, we consider the problem of anomaly detection when a small number of anomaly types are observed. Previous research primarily focused on supervised learning, when all samples are labeled. And unsupervised learning is used, when all samples are unlabeled. However, many settings do not satisfy the above two situations. Recently, there are some studies on the situations that anomalies are partially observed (e.g., Anomaly Detection with partially Observed Anomalies). It is generally believed that the anomalies are classifiable in these studies. And it is common that the types of observed anomalies cannot include all types of anomalies in the case of partially observed anomalies. We refer to this problem as anomaly detection with partially observed anomaly types and propose a two-stage anomaly detection algorithm in this condition. The proposed method in this paper is based on Anomaly Detection with partially Observed Anomalies and is available in the new setting. Experimental results demonstrate the effectiveness of the proposed method both in the case of insufficient types of observed anomalies and in the case of sufficient types of observed anomalies. Besides, anomaly detection with partially observed anomaly types avoids the use of hyper-parameter and has high generality in different datasets.
Anomaly (physics)
Generality
Cite
Citations (1)
Most anomaly detection systems try to model normal behavior and assume anomalies deviate from it in diverse manners. However, there may be patterns in the anomalies as well. Ideally, an anomaly detection system can exploit patterns in both normal and anomalous behavior. In this paper, we present AD-MERCS, an unsupervised approach to anomaly detection that explicitly aims at doing both. AD-MERCS identifies multiple subspaces of the instance space within which patterns exist, and identifies conditions (possibly in other subspaces) that characterize instances that deviate from these patterns. Experiments show that this modeling of both normality and abnormality makes the anomaly detector performant on a wide range of types of anomalies. Moreover, by identifying patterns and conditions in (low-dimensional) subspaces, the anomaly detector can provide simple explanations of why something is considered an anomaly. These explanations can be both negative (deviation from some pattern) as positive (meeting some condition that is typical for anomalies).
Anomaly (physics)
Abnormality
Cite
Citations (0)
Anomaly detectors are used to distinguish the difference between normal and abnormal data, which are usually implemented by evaluating and ranking anomaly scores of each instance. Static unsupervised anomaly detectors can be difficult to adjust anomaly score calculation for streaming data. In real scenarios, anomaly detection often needs to be regulated by human feedback, which benefits to adjust anomaly detectors. In this paper, we propose a human-machine interactive anomaly detection method, named ISPForest, which can be adaptively updated under the guidance of human feedback. In particular, the feedback will be used to adjust the anomaly score calculation and structure of the tree-based detector, ideally attaining more accurate anomaly scores in the future. Our main contribution is to improve the tree model that can be dynamically updated from perspectives of anomaly score calculation and the model’s structure. Our approach is instantiated for the powerful class of tree-based anomaly detectors, and we conduct experiments on a range of benchmark datasets. The results demonstrate that human expert feedback is helpful to improve the accuracy of anomaly detectors.
Anomaly (physics)
Benchmark (surveying)
Tree (set theory)
Cite
Citations (1)
This abstract proposes a time series anomaly detector which 1) makes no assumption about the underlying mechanism of anomaly patterns, 2) refrains from the cumbersome work of threshold setting for good anomaly detection performance under specific scenarios, and 3) keeps evolving with the growth of anomaly detection experience. Essentially, the anomaly detector is powered by the Recurrent Neural Network (RNN) and adopts the Reinforcement Learning (RL) method to achieve the self-learning process. Our initial experiments demonstrate promising results of using the detector in network time series anomaly detection problems.
Anomaly (physics)
Cite
Citations (48)
We propose a supervised anomaly detection method for data with inexact anomaly labels, where each label, which is assigned to a set of instances, indicates that at least one instance in the set is anomalous. Although many anomaly detection methods have been proposed, they cannot handle inexact anomaly labels. To measure the performance with inexact anomaly labels, we define the inexact AUC, which is our extension of the area under the ROC curve (AUC) for inexact labels. The proposed method trains an anomaly score function so that the smooth approximation of the inexact AUC increases while anomaly scores for non-anomalous instances become low. We model the anomaly score function by a neural network-based unsupervised anomaly detection method, e.g., autoencoders. The proposed method performs well even when only a small number of inexact labels are available by incorporating an unsupervised anomaly detection mechanism with inexact AUC maximization. Using various datasets, we experimentally demonstrate that our proposed method improves the anomaly detection performance with inexact anomaly labels, and outperforms existing unsupervised and supervised anomaly detection and multiple instance learning methods.
Anomaly (physics)
Data set
Maximization
Cite
Citations (0)
Automatic anomaly detection for time series is essential for supervising the system operation and diagnosing abnormal events in the spacecraft system. Many anomaly detection approaches have been proposed in recent years. However, most anomaly detectors give scores to the detected anomalies solely according to the information collected during the learning and detection processes, which may not be dependable. As a result, when a large number of false alarms are detected, it is difficult to prune them using the given anomaly scores without sacrificing correctly detected true anomalies. In this paper, we propose a post-detection verification method based on a fast and accurate time series subsequence matching algorithm. Given a detected anomaly, we find its top-k most similar subsequences from the normal dataset (sequences assumed to be anomaly-free). Then a distance score is calculated for the detected anomaly. Also, P subsequences with the same length as the detected anomaly are extracted from the normal part (the part with no detected anomalies) of the test sequence. The distance scores of these P subsequences with respect to their closest counterparts in the normal dataset are calculated. Finally, we compare the distance score of the detected anomaly with the distance scores of the P normal subsequences to verify or reject it as a true anomaly. To evaluate the proposed method, we have created a challenging dataset MRO-SIN by injecting anomalies into the Mars Re-connaissance Orbiter (MRO) dataset, to allow for quantitative assessment. A stacked-predictor-based anomaly detector generates many false alarms and an Fl score of 0.512 on the MRO-SIN dataset. The new anomaly verification method significantly reduces the number of false alarms and improves the Fl score from 0.512 to 0.676.
Anomaly (physics)
Cite
Citations (2)
In this study, we proposed a data-driven approach to the condition monitoring of the marine engine. Although several unsupervised methods in the maritime industry have existed, the common limitation was the interpretation of the anomaly; they do not explain why the model classifies specific data instances as an anomaly. This study combines explainable AI techniques with anomaly detection algorithm to overcome the limitation above. As an explainable AI method, this study adopts Shapley Additive exPlanations (SHAP), which is theoretically solid and compatible with any kind of machine learning algorithm. SHAP enables us to measure the marginal contribution of each sensor variable to an anomaly. Thus, one can easily specify which sensor is responsible for the specific anomaly. To illustrate our framework, the actual sensor stream obtained from the cargo vessel collected over 10 months was analyzed. In this analysis, we performed hierarchical clustering analysis with transformed SHAP values to interpret and group common anomaly patterns. We showed that anomaly interpretation and segmentation using SHAP value provides more useful interpretation compared to the case without using SHAP value.
Anomaly (physics)
Cite
Citations (34)
On the issue of low precision of ship anomaly behavior detection method based on global variable and calculation complexity of ship anomaly detection based on local variable, a combination of K Nearest Neighbor (KNN) and Local Outlier Factor (LOF) algorithm for ship anomaly behavior detection is proposed in this paper. Firstly, ship anomaly data candidate set is filtered by K nearest neighbor, then calculating local deviation index by LOF algorithm, lastly setting threshold value to judge ship anomaly behavior, so as to achieve rapid, effective ship anomaly behavior detection. To a certain extent, it helps the maritime safety supervision department to identify the potential risks of their ship, and improve regulatory efficiency.
Local outlier factor
Anomaly (physics)
Identification
Data set
Cite
Citations (11)