Similar to the encoder, the decoder also uses LSTM layers, taking the context vector and a previous word (during training) to predict the next word in the summary. The decoder ends with a softmax ...

This project is a text summarization tool built entirely from scratch using an LSTM-based encoder-decoder architecture. While it has been trained on a toy dataset and is limited in performance, it is ...

In order to overcome the drawback of decoder-only LLMs for text embedding, a team of researchers from Mila, McGill University, ServiceNow Research, and Facebook CIFAR AI Chair has proposed LLM2Vec, a ...

The data-to-text generation task mainly uses the encoder-decoder architecture, in which the context module provides the information that the decoder wants to observe at the moment. However, there are ...

Dense retrieval requires high-quality text sequence embeddings to support effective search in the representation space. Autoencoder-based language models are appealing in dense retrieval as they train ...

The rapid development of single-modal pre-training has prompted researchers to pay more attention to cross-modal pre-training methods. In this paper, we propose a unified-modal speech-unit-text ...