On error of value function inevitably causes an overestimation phenomenon and has a negative impact on the convergence of the algorithms. To mitigate the negative effects of the approximation error, we propose Error Controlled Actor-critic which ensures confining the approximation error in value function. We present an analysis of how the approximation error can hinder the optimization process of actor-critic methods.Then, we derive an upper boundary of the approximation error of Q function approximator and find that the error can be lowered by restricting on the KL-divergence between every two consecutive policies when training the policy. The results of experiments on a range of continuous control tasks demonstrate that the proposed actor-critic algorithm apparently reduces the approximation error and significantly outperforms other model-free RL algorithms.
In the field of crowd-counting research, many recent deep learning based methods have demonstrated robust capabilities for accurately estimating crowd sizes. However, the enhancement in their performance often arises from an increase in the complexity of the model structure. This paper introduces the Fuss-Free Network (FFNet), a crowd counting deep learning model that is characterized by its simplicity and efficiency in terms of its structure. The model comprises only a backbone of a neural network and a multi-scale feature fusion structure.The multi-scale feature fusion structure is a simple architecture consisting of three branches, each only equipped with a focus transition module, and combines the features from these branches through the concatenation operation.Our proposed crowd counting model is trained and evaluated on four widely used public datasets, and it achieves accuracy that is comparable to that of existing complex models.The experimental results further indicate that excellent performance in crowd counting tasks can also be achieved by utilizing a simple, low-parameter, and computationally efficient neural network structure.
Peripheral blood cell detection is an essential component of medical practice and is used to diagnose and treat diseases, as well as to monitor the progress of therapies. Our objective is to construct an efficient deep learning model for peripheral blood cell analysis that achieves an optimized balance between inference speed, computational complexity, and detection accuracy. In this article, we propose the DWS-YOLO blood detector, which is a lightweight blood detector. Our model includes several improved modules, including the lightweight C3 module, the increased combined attention mechanism, the Scylla-IoU loss function, and the improved soft non-maximum suppression. Improved attention, loss function, and suppression enhance detection accuracy, while lightweight C3 module reduces computation time. The experiment results demonstrate that our proposed modules can enhance a detector’s detection performance, and obtain new state-of-the-art (SOTA) results and excellent robustness performance on the BCCD dataset. On the white blood cell detection dataset (Raabin-WBC), the proposed detector’s generalization performance was confirmed to be satisfactory. Our proposed blood detector achieves high detection accuracy while requiring few computational resources and is very suitable for resource-limited but efficient medical device environments, providing a reliable and advanced solution for blood detection that greatly improves the efficiency and effectiveness of peripheral blood cell analysis in clinical practice.
With the increasing aging of the population, cardiovascular diseases have become one of the most important causes of death. Arrhythmia is one of the common conditions of cardiovascular diseases, and electrocardiogram (ECG) can reflect information about the electrical activity of the heart, and an abnormal ECG can detect different kinds of arrhythmia. By visual inspection of ECG is not only time consuming but can lead to misdiagnosis which in turn affects the treatment of the disease. With the enhanced availability of wearable devices, home monitoring turns out to be a more preferred option for the elderly. In this paper, a lightweight automatic arrhythmia classification method based on wavelet soft threshold transform method and convolutional neural network (CNN) is proposed. Experiments were conducted using the MIT-BIH arrhythmia dataset for the training and validation of the model, and arrhythmias were categorized into five categories according to the Association for the Advancement of Medical Instrumentation (AAMI) criteria, and an accuracy of 99.47% was finally obtained.
Surface Enhanced Raman Spectroscopy (SERS) has been widely used as a sensitive sensing technology in the medical field because of its unique molecular fingerprint information. In this paper, silver nanospheres (Ag NPs) and silver nano-cube (Ag NC) nanoparticles with different morphologies were constructed based on silver nanomaterials for the purpose of early screening of kidney cancer. By analyzing the SERS characteristics of nanoparticles, it was found that the enhancement effect of Ag NC was greater than that of Ag NPs. SERS detection was performed on the urine of 30 patients with kidney cancer and 30 normal subjects. By analyzing the spectral differences between cancer patients and normal people, the cancer group and normal people were preliminarily distinguished. We analyzed the measured spectral data. The analysis methods mainly included Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) diagnostic algorithms, as well as recursive weighted Partial Least Squares (PLS) and Support Vector Machine (SVM) algorithms. The comparison of the two classification algorithms shows that the classification accuracy of PLS-SVM is 99.13%, sensitivity was 98.67%, specificity was 100%, and AUC value was 1. The classification effect was much higher than PCA-LDA. The results of this Exploratory research show that the combination of Ag NC substrate and PLS-SVM algorithm has greater potential in the pre diagnosis and screening of Kidney cancer.
Learning how to write Chinese character strokes from the stroke images directly, has a great significance to the inheritance of calligraphy art and to imitate the writing style of Chinese calligraphers. However, most of the existing methods directly applied existing samples with action labels. The performance of these methods is often limited by the quality and number of samples. Thus, these methods cannot be used to learn calligraphy from unlabeled samples. To address this problem, a calligraphy robotic model based on deep reinforcement learning is proposed in this paper, which enables a robotic arm to write fundamental Chinese character strokes from stroke images. In the model, writing task is seen as the process of interaction between the robot and the environment. The robot makes appropriate writing action based on the state information provided by the environment. In order to evaluate the writing action of the robot, a reward function is designed on the model. In addition, the stochastic policy gradient method is used in training on the model. Finally, the model was extensively experimented on a stroke data set. Environmental results demonstrate that the proposed model allows a calligraphy robot to successfully write fundamental Chinese character strokes from stroke images. This model provides a promising solution for reconstructing writing actions from images.
Intelligent robots are required to fully understand human intentions and operations in order to support or collaborate with humans to complete complicated tasks, which is typically implemented by employing human-machine interaction techniques.This paper proposes a new robotic learning framework to perform numeral writing tasks by investigating human-machine interactions with human preferences.In particular, the framework implements a trajectory generative module using a generative adversarial network (GAN)-based method and develops a human preference feedback system to enable the robot to learn human preferences.In addition, a convolutional neural network, acting as a discriminative network, classifies numeral images to support the development of the basic numeral writing ability, and another convolutional neural network, acting as a human preference network, learns a human user's aesthetic preference by taking the feedback on two written numerical images during the training process.The experimental results show that the written numerals based on the preferences of ten users were different from those of the training data set and that the writing models with the preferences from different users generate numerals in different styles, as evidenced by the Fréchet inception distance (FID) scores.The FID scores of the proposed framework with a preference network were noticeably greater than those of the framework without a preference network.This phenomenon indicates that the human-machine interactions effectively guided the robotic system to learn different writing styles.These results prove that the proposed approach is able to enable the calligraphy robot to successfully write numerals in accordance with the preferences of a human user.
'/%3e%3cuse xlink:href='%23a' transform='rotate(23.036 .093 25.536)'/%3e%3cuse xlink:href='%23a' transform='rotate(45.87 1.273 16.18)'/%3e%3cuse xlink:href='%23a' transform='rotate(69.945 .996 12.078)'/%3e%3cuse xlink:href='%23a' transform='rotate(20.66 -19.689 31.932)'/%3e%3c/svg%3e)
'%3e%3cpath fill='%23012169' d='M0 0v30h60V0z'/%3e%3cpath stroke='%23fff' stroke-width='6' d='m0 0 60 30m0-30L0 30'/%3e%3cpath stroke='%23C8102E' stroke-width='4' d='m0 0 60 30m0-30L0 30' clip-path='url(%23b)'/%3e%3cpath stroke='%23fff' stroke-width='10' d='M30 0v30M0 15h60'/%3e%3cpath stroke='%23C8102E' stroke-width='6' d='M30 0v30M0 15h60'/%3e%3c/g%3e%3c/svg%3e)