SSA: A Content-Based Sparse Attention Mechanism

Yang Sun; Wei Hu; Fang Liu; Feihu Huang; Yonghao Wang

doi:10.1007/978-3-031-10989-8_53

SSA: A Content-Based Sparse Attention Mechanism

Yang Sun, Wei Hu^*, Fang Liu, Feihu Huang, Yonghao Wang

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Citations (SciVal)

Original language	English
Title of host publication	Knowledge Science, Engineering and Management - 15th International Conference, KSEM 2022, Proceedings
Editors	Gerard Memmi, Baijian Yang, Linghe Kong, Tianwei Zhang, Meikang Qiu
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	669-680
Number of pages	12
ISBN (Print)	9783031109881
DOIs	https://doi.org/10.1007/978-3-031-10989-8_53
Publication status	Published (VoR) - 2022
Event	15th International Conference on Knowledge Science, Engineering and Management, KSEM 2022 - Singapore, Singapore Duration: 6 Aug 2022 → 8 Aug 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13370 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	15th International Conference on Knowledge Science, Engineering and Management, KSEM 2022
Country/Territory	Singapore
City	Singapore
Period	6/08/22 → 8/08/22

Keywords

Global attention
Local attention
Sequence
Sparsity
Transformers

Access to Document

10.1007/978-3-031-10989-8_53

Cite this

Sun, Y., Hu, W., Liu, F., Huang, F., & Wang, Y. (2022). SSA: A Content-Based Sparse Attention Mechanism. In G. Memmi, B. Yang, L. Kong, T. Zhang, & M. Qiu (Eds.), Knowledge Science, Engineering and Management - 15th International Conference, KSEM 2022, Proceedings (pp. 669-680). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13370 LNAI). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-10989-8_53

Sun, Yang ; Hu, Wei ; Liu, Fang et al. / SSA : A Content-Based Sparse Attention Mechanism. Knowledge Science, Engineering and Management - 15th International Conference, KSEM 2022, Proceedings. editor / Gerard Memmi ; Baijian Yang ; Linghe Kong ; Tianwei Zhang ; Meikang Qiu. Springer Science and Business Media Deutschland GmbH, 2022. pp. 669-680 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{c41a3770f47348a89200d4a65b28bbda,

title = "SSA: A Content-Based Sparse Attention Mechanism",

keywords = "Global attention, Local attention, Sequence, Sparsity, Transformers",

author = "Yang Sun and Wei Hu and Fang Liu and Feihu Huang and Yonghao Wang",

note = "Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 15th International Conference on Knowledge Science, Engineering and Management, KSEM 2022 ; Conference date: 06-08-2022 Through 08-08-2022",

year = "2022",

doi = "10.1007/978-3-031-10989-8_53",

language = "English",

isbn = "9783031109881",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "669--680",

editor = "Gerard Memmi and Baijian Yang and Linghe Kong and Tianwei Zhang and Meikang Qiu",

booktitle = "Knowledge Science, Engineering and Management - 15th International Conference, KSEM 2022, Proceedings",

}

Sun, Y, Hu, W, Liu, F, Huang, F & Wang, Y 2022, SSA: A Content-Based Sparse Attention Mechanism. in G Memmi, B Yang, L Kong, T Zhang & M Qiu (eds), Knowledge Science, Engineering and Management - 15th International Conference, KSEM 2022, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13370 LNAI, Springer Science and Business Media Deutschland GmbH, pp. 669-680, 15th International Conference on Knowledge Science, Engineering and Management, KSEM 2022, Singapore, Singapore, 6/08/22. https://doi.org/10.1007/978-3-031-10989-8_53

SSA: A Content-Based Sparse Attention Mechanism. / Sun, Yang; Hu, Wei; Liu, Fang et al.
Knowledge Science, Engineering and Management - 15th International Conference, KSEM 2022, Proceedings. ed. / Gerard Memmi; Baijian Yang; Linghe Kong; Tianwei Zhang; Meikang Qiu. Springer Science and Business Media Deutschland GmbH, 2022. p. 669-680 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13370 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - SSA

T2 - 15th International Conference on Knowledge Science, Engineering and Management, KSEM 2022

AU - Sun, Yang

AU - Hu, Wei

AU - Liu, Fang

AU - Huang, Feihu

AU - Wang, Yonghao

PY - 2022

Y1 - 2022

KW - Global attention

KW - Local attention

KW - Sequence

KW - Sparsity

KW - Transformers

UR - http://www.scopus.com/inward/record.url?scp=85135099300&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85135099300&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-10989-8_53

DO - 10.1007/978-3-031-10989-8_53

M3 - Conference contribution

AN - SCOPUS:85135099300

SN - 9783031109881

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 669

EP - 680

BT - Knowledge Science, Engineering and Management - 15th International Conference, KSEM 2022, Proceedings

A2 - Memmi, Gerard

A2 - Yang, Baijian

A2 - Kong, Linghe

A2 - Zhang, Tianwei

A2 - Qiu, Meikang

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 6 August 2022 through 8 August 2022

ER -

Sun Y, Hu W, Liu F, Huang F, Wang Y. SSA: A Content-Based Sparse Attention Mechanism. In Memmi G, Yang B, Kong L, Zhang T, Qiu M, editors, Knowledge Science, Engineering and Management - 15th International Conference, KSEM 2022, Proceedings. Springer Science and Business Media Deutschland GmbH. 2022. p. 669-680. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-10989-8_53

SSA: A Content-Based Sparse Attention Mechanism

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this