Abstract Background: Esophageal cancer (ESCC) is one of the most common malignant tumors in the digestive system. This study aims to explore the effects of sperm associated antigen 5 (SPAG5) on cell growth, metastasis, and azithromycin resistance in esophagus cancer and its molecular mechanism. Methods: The DEGs were obtained from GSE92396, GSE17351, and GSE9982 datasets about ESCC. The PPI network was constructed using the STRING database and was visualized using Cytoscape software. The cytohubba plug-in of Cytoscape software was used to identify the hub genes of the PPI network. The DEGs were used to perform GO and KEGG pathway enrichment analysis using the DAVID database. Statistical analysis was performed to test the clinical and prognostic significance of SPAG5. Cell viability, proliferation, apoptosis, migration, and invasion were detected using CCK-8, colony formation, flow cytometry, transwell and scratch-wound assays. The expression of related genes was detected by qRT-PCR, western blot and IHC assays. The oncogenicity of SPAG5 in ESCC cells was determined using the nude mouse transplantation tumor experiment. Results: Ninety-three overlapping genes from the DEGs were used to construct the PPI network, and mainly enriched in BP, CC, and MF terms. COX regression analysis of OS showed that SPAG5 expression and pN category were correlated with OS. Univariate and multivariate analyses showed that SPAG5 was an independent prognostic factor for OS in ESCC. The ROC curve analysis showed the AUC of SPAG5 was 0.74. Multiple logistic regression showed that SPAG5 were subsequently identified as an independent risk factor associated with OS. SPAG5 overexpression was detected in ESCC tissues and cell lines, and improved cell proliferation. SPAG5 knockdown reduced cell growth and metastasis and promoted its apoptosis. The functions of SPAG5 overexpression promoting ESCC cell growth and affecting cleaved caspase-3, Ki67, VEGF, and MMP-2/-9 expression were reversed by PI3K/AKT inhibitor. SPAG5 overexpression enhanced resistance to ADM in EC9706 and Eca109 cells and it was closely related to the activation of PI3K/AKT signaling pathway. Conclusion: The overexpression of SPAG5 was an independent good prognostic factor and promoted the proliferation, invasion, migration, and ADM resistance, and inhibited the apoptosis via activating PI3K/AKT signaling pathway in ESCC.
Recent works have shown that powerful pre-trained language models (PLM) can be fooled by small perturbations or intentional attacks. To solve this issue, various data augmentation techniques are proposed to improve the robustness of PLMs. However, it is still challenging to augment semantically relevant examples with sufficient diversity. In this work, we present Virtual Data Augmentation (VDA), a general framework for robustly fine-tuning PLMs. Based on the original token embeddings, we construct a multinomial mixture for augmenting virtual data embeddings, where a masked language model guarantees the semantic relevance and the Gaussian noise provides the augmentation diversity. Furthermore, a regularized training strategy is proposed to balance the two aspects. Extensive experiments on six datasets show that our approach is able to improve the robustness of PLMs and alleviate the performance degradation under adversarial attacks. Our codes and data are publicly available at bluehttps://github.com/RUCAIBox/VDA.
Sentence ordering aims to arrange the sentences of a given text in the correct order. Recent work frames it as a ranking problem and applies deep neural networks to it. In this work, we propose a new method, named BERT4SO, by fine-tuning BERT for sentence ordering. We concatenate all sentences and compute their representations by using multiple special tokens and carefully designed segment (interval) embeddings. The tokens across multiple sentences can attend to each other which greatly enhances their interactions. We also propose a margin-based listwise ranking loss based on ListMLE to facilitate the optimization process. Experimental results on five benchmark datasets demonstrate the effectiveness of our proposed method.
Objective To investigate the effect of bradykinin (BK) on human lung cancer cell line A549 migration,invasion ability. Methods Scratch Healing and Transwell assay of bradykinin were used to detect scratch healing and movement on A549 human lung cancer cells. Analysis the changes of matrix metalloproteinase (MMP)-2, MMP-9 and E-Cadherin after stimulate and suppress 20 cases respectively in the aim to describe the mechanism by Western blotting. Results In the former, BK group promote cell migration significantly,migrated cell was 69.2±3.3, 94.1±2.9 respectively after 24 h and 48 h,blocking its receptor or ERK1/2 pathway inhibit migration,migrated cell was 51.2±2.1, 73.2±2.7 and 47.5±3.4,77.6±3.8 after 24 h and 48 h. In the latter,Transwell assay of A549 BK showed invasion of A549 BK can significantly promote the invasion ability of A549 to penetrate the basement membrane of the cell. And blocking BK receptor or ERK1/2 pathway can inhibit the invasion. Conclusion Bradykinin promote human lung adenocarcinoma cell line A549 migration and invasion.
Key words:
Bradykinin; Lung cancer cells; Migration; Invasion
Within online platforms, it is critical to capture the semantics of sequential user behaviors for accurately modeling user interests. However, dynamic characteristics and sparse behaviors make it difficult to train effective user representations for sequential user behavior modeling.
Mathematical reasoning is an important capability of large language models~(LLMs) for real-world applications. To enhance this capability, existing work either collects large-scale math-related texts for pre-training, or relies on stronger LLMs (\eg GPT-4) to synthesize massive math problems. Both types of work generally lead to large costs in training or synthesis. To reduce the cost, based on open-source available texts, we propose an efficient way that trains a small LLM for math problem synthesis, to efficiently generate sufficient high-quality pre-training data. To achieve it, we create a dataset using GPT-4 to distill its data synthesis capability into the small LLM. Concretely, we craft a set of prompts based on human education stages to guide GPT-4, to synthesize problems covering diverse math knowledge and difficulty levels. Besides, we adopt the gradient-based influence estimation method to select the most valuable math-related texts. The both are fed into GPT-4 for creating the knowledge distillation dataset to train the small LLM. We leverage it to synthesize 6 million math problems for pre-training our JiuZhang3.0 model, which only needs to invoke GPT-4 API 9.3k times and pre-train on 4.6B data. Experimental results have shown that JiuZhang3.0 achieves state-of-the-art performance on several mathematical reasoning datasets, under both natural language reasoning and tool manipulation settings. Our code and data will be publicly released in \url{https://github.com/RUCAIBox/JiuZhang3.0}.
Recent years have witnessed the boom of deep neural networks in online news recommendation service. As news articles mainly consist of textual content, pre-trained language models~(PLMs) (e.g. BERT) have been widely adopted as the backbone to encode them into news embeddings, which would be utilized to generate the user representations or perform the semantic matching. However, existing PLMs are mostly pre-trained on large-scale general corpus, and have not been specially adapted for capturing the rich information within news articles. Therefore, their produced news embeddings may be not informative enough to represent the news content or characterize the relations among news. To solve it, we propose a bottlenecked multi-task pre-training approach, which relies on an information-bottleneck encoder-decoder architecture to compress the useful semantic information into the news embedding. Concretely, we design three pre-training tasks, to enforce the news embedding to recover the news contents of itself, its frequently oc-occurring neighbours, and the news with similar topics. We conduct experiments on the MIND dataset and show that our approach can outperform competitive pre-training methods.
Recently, significant progress has been made in sequential recommendation with deep learning. Existing neural sequential recommendation models usually rely on the item prediction loss to learn model parameters or data representations. However, the model trained with this loss is prone to suffer from data sparsity problem. Since it overemphasizes the final performance, the association or fusion between context data and sequence data has not been well captured and utilized for sequential recommendation. To tackle this problem, we propose the model S^3-Rec, which stands for Self-Supervised learning for Sequential Recommendation, based on the self-attentive neural architecture. The main idea of our approach is to utilize the intrinsic data correlation to derive self-supervision signals and enhance the data representations via pre-training methods for improving sequential recommendation. For our task, we devise four auxiliary self-supervised objectives to learn the correlations among attribute, item, subsequence, and sequence by utilizing the mutual information maximization (MIM) principle. MIM provides a unified way to characterize the correlation between different types of data, which is particularly suitable in our scenario. Extensive experiments conducted on six real-world datasets demonstrate the superiority of our proposed method over existing state-of-the-art methods, especially when only limited training data is available. Besides, we extend our self-supervised learning method to other recommendation models, which also improve their performance.
In recent years, conversational recommender system (CRS) has received much attention in the research community. However, existing studies on CRS vary in scenarios, goals and techniques, lacking unified, standardized implementation or comparison. To tackle this challenge, we propose an open-source CRS toolkit CRSLab, which provides a unified and extensible framework with highly-decoupled modules to develop CRSs. Based on this framework, we collect 6 commonly-used human-annotated CRS datasets and implement 18 models that include recent techniques such as graph neural network and pre-training models. Besides, our toolkit provides a series of automatic evaluation protocols and a human-machine interaction interface to test and compare different CRS methods. The project and documents are released at https://github.com/RUCAIBox/CRSLab.
'/%3e%3cuse xlink:href='%23a' transform='rotate(23.036 .093 25.536)'/%3e%3cuse xlink:href='%23a' transform='rotate(45.87 1.273 16.18)'/%3e%3cuse xlink:href='%23a' transform='rotate(69.945 .996 12.078)'/%3e%3cuse xlink:href='%23a' transform='rotate(20.66 -19.689 31.932)'/%3e%3c/svg%3e)
'%3e%3cpath fill='%23012169' d='M0 0v30h60V0z'/%3e%3cpath stroke='%23fff' stroke-width='6' d='m0 0 60 30m0-30L0 30'/%3e%3cpath stroke='%23C8102E' stroke-width='4' d='m0 0 60 30m0-30L0 30' clip-path='url(%23b)'/%3e%3cpath stroke='%23fff' stroke-width='10' d='M30 0v30M0 15h60'/%3e%3cpath stroke='%23C8102E' stroke-width='6' d='M30 0v30M0 15h60'/%3e%3c/g%3e%3c/svg%3e)