cross-attention

noun

noun

Etymology From cross- + attention.

A form of attention (machine learning method) where two different input sequences are compared, i.e. the keys and queries differ.

Machine learning

The cross attention layer inside each transformer decoder block is in fact a multi-head cross attention layer […]2024, Benoit Liquet, Sarat Moka, Yoni Nazarathy, Mathematical Engineering of Deep Learning, CRC Press, page 286:

Entry derived from the Wiktionary, under licence CC BY-SA 4.0 — list of authors.