Hi guys, Welcome to the Advanced Video Search Engine, and you are watching Adaptive Transformers in NLP. That video is uploaded by AI Journal at 2020-05-12T07:25:29-07:00. We are promoting this video only for entertainment and educational purpose only. So, I hope you like & enjoy it. [_pYxa50HTBw]
| Name | Adaptive Transformers in NLP |
| Video Uploader | Video From AI Journal |
| Upload Date | This Video Uploaded At 12-05-2020 14:25:29 |
| Video Discription | We discuss numerous areas of research in NLP focussed upon, making deep learning models lighter, reducing computation (FLOPS), inducing sparsity, and other related methods. Specifically, we discuss, including Pruning, Quantization, Distillation, and Adaptive Methods. We discuss the following papers in detail: - Adaptive Attention Span (https://arxiv.org/abs/1905.07799) - Adaptively Sparse Transformers (https://arxiv.org/abs/1909.00015) - Reducing Transformer Depth on Demand with Structured Dropout (https://arxiv.org/abs/1909.11556) 0:00 Introduction 0:32 Methods to reduce computation 0:37 Pruning 0:58 Sparse networks (Lottery Tickets Hypothesis) 1:32 Quantization 1:59 Knowledge Distillation 2:20 Adaptive Methods: Introduction 3:20 Adaptive Attention Span 6:15 Adaptively Sparse Transformers 10:33 Reducing Transformer Depth on Demand With Structured Dropout (Layerdrop) 13:03 End credits Github repository: https://github.com/prajjwal1/adaptive_transformer Fluence: https://github.com/prajjwal1/fluence Connect with me on Twitter: https://twitter.com/prajjwal_1 Please support me on Patreon : https://patreon.com/aijournal _pYxa50HTBw |
| Category | Education |
| Tags | transformers | bert | adaptive transformer | language model | roberta | attention is all you need | transformer | adaptive attention span | layerdrop | sparse | sparsity | sparse neural network | dropout | bidirectional | gpt2 | openai | deep learning | nlp | natural language processing | autoregressive | seq2seq | recurrent | tensor2tensor | fairseq | pytorch | machine translation | attention | attention transformer | attention neural networks | transformer neural network | facebook ai research |
