An Attentive Survey of Attention Models

doi:10.1145/3465055

Paper

An Attentive Survey of Attention Models

Published Apr 5, 2019 · S. Chaudhari, Gungor Polatkan, R. Ramanath

ACM Transactions on Intelligent Systems and Technology (TIST)

Q1 SJR score

581

Citations

22

Influential Citations

Full text Semantic Scholar

Abstract

Attention Model has now become an important concept in neural networks that has been researched within diverse application domains. This survey provides a structured and comprehensive overview of the developments in modeling attention. In particular, we propose a taxonomy that groups existing techniques into coherent categories. We review salient neural architectures in which attention has been incorporated and discuss applications in which modeling attention has shown a significant impact. We also describe how attention has been used to improve the interpretability of neural networks. Finally, we discuss some future research directions in attention. We hope this survey will provide a succinct introduction to attention models and guide practitioners while developing approaches for their applications.

Highly Cited

Study Snapshot

Key takeawayThis survey provides a comprehensive overview of attention models in neural networks, introducing a taxonomy and guiding practitioners in developing attention-based approaches for various applications.

PopulationOlder adults (50-71 years)

Sample size24

MethodsObservational

OutcomesBody Mass Index projections

ResultsSocial networks mitigate obesity in older groups.

An Attentive Survey of Attention Models

References

Citations