In the context of attention, there is a very interesting recent paper that warns... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

abhgh on Sept 16, 2019 | parent | context | favorite | on: Attention Mechanism in Deep Learning

In the context of attention, there is a very interesting recent paper that warns against conflating attention and token importance - "Is Attention Interpretable?" [1]. This is an accepted paper in ACL-2019:

[1] https://www.aclweb.org/anthology/P19-1282

stochastic_monk on Sept 16, 2019 [–]

See also Attention is Not Explanation [0].

[0] https://www.aclweb.org/anthology/N19-1357

physicsyogi on Sept 17, 2019 | | [–]

There's a rebuttal to this as well: Attention is not not Explanation. https://arxiv.org/abs/1908.04626

abhgh on Sept 16, 2019 | | [–]

Thanks!

Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact