CRM

The WikiText Long Term Dependency Language Modeling Dataset

The WikiText language modeling dataset is a collection of over 100 million tokens extracted from the set of verified Good and Featured articles on Wikipedia. The dataset is available under the Creative Commons Attribution-ShareAlike License. Compared to the preprocessed version of Penn Treebank (PTB), WikiText-2 is over 2 times larger and WikiText-103 is over 110 […]

The WikiText Long Term Dependency Language Modeling Dataset Read More »

Teaching Neural Networks to Point to Improve Language Modeling and Translation

Imagine you were a young child and wanted to ask about something. Being so young (and assuming you are not exceedingly precocious), how would you describe a new object, the name of which you have yet to learn? The intuitive answer: point to it! Surprisingly, neural networks have the same issue. Neural Networks typically use

Teaching Neural Networks to Point to Improve Language Modeling and Translation Read More »

New Neural Network Building Block Allows Faster and More Accurate Text Understanding

In deep learning, there are two very different ways to process input (like an image or a document). Typically, images are processed all at once, with the same kind of computation happening for every part of the image simultaneously. But researchers have usually assumed that you can’t do this for text data: that you need

New Neural Network Building Block Allows Faster and More Accurate Text Understanding Read More »

State of the art Deep Learning Model for Question Answering

We introduce the Dynamic Coattention Network, a state of the art neural network designed to automatically answer questions about documents. Instead of producing a single, static representation of the document without context, our system is able to interpret the document differently depending on the question. That is, given the same document, the system constructs a

State of the art Deep Learning Model for Question Answering Read More »

Multiple Different Natural Language Processing Tasks in a Single Deep Model

Humans learn natural languages, such as English, starting from basic grammar to complex semantics in a single brain. How do we build such a single model to handle a variety of natural language processing (NLP) tasks in computers? As a first step towards this goal, we have developed a single deep neural network model which

Multiple Different Natural Language Processing Tasks in a Single Deep Model Read More »

A way out of the Odyssey: Analyzing and Combining Recent Insights for LSTMs

LSTMs have become a basic building block for many deep NLP models. In recent years, many improvements and variations have been proposed for deep sequence models in general, and LSTMs in particular. We propose and analyze a series of augmentations and modifications to LSTM networks resulting in improved performance for text classification datasets. We observe

A way out of the Odyssey: Analyzing and Combining Recent Insights for LSTMs Read More »

Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning

Automatically generating captions for images has emerged as a prominent interdisciplinary research problem in both academia and industry. It can aid visually impaired users, and make it easy for users to organize and navigate through large amounts of typically unstructured visual data. In order to generate high quality captions, the model needs to incorporate fine-grained

Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning Read More »

Thinking out Loud: Hierarchical and Interpretable Multi-task Reinforcement Learning

Deep reinforcement learning (deep RL) is a popular and successful family of methods for teaching computers tasks ranging from playing Go and Atari games to controlling industrial robots. But it is difficult to use a single neural network and conventional RL techniques to learn many different skills at once. Existing approaches usually treat the tasks

Thinking out Loud: Hierarchical and Interpretable Multi-task Reinforcement Learning Read More »

Improving end-to-end Speech Recognition Models

Speech recognition has been successfully depolyed on various smart devices, and is changing the way we interact with them. Traditional phonetic-based recognition approaches require training of separate components such as pronouciation, acoustic and language model. Since the models are all trained separately with different training objectives, improving one of the components does not necessarily lead

Improving end-to-end Speech Recognition Models Read More »