alignment tax
nounEtymology First attested in a speech in 2019 by computer scientist Paul Christiano (see quotation), who attributed the idea to AI researcher and writer Eliezer Yudkowsky.
1
A cost to the capabilities of an artificial intelligence resulting from the effects of aligning it with human ethics and morality.
Artificial intelligence
- I like this notion of an "alignment tax" […] the reason I might compromise is if there's some tension, between having the AI that's robustly trying to do what I want, and having the AI that is competent or intelligent, and the alignment tax is intended to capture that gap—that cost that I incur if I insist on alignment.2019 August 29, Paul Christiano, Current work in AI alignment, EA Global San Francisco 2019:
- The fact that larger models are less subject to forgetting may be related to the fact that larger models do not incur significant alignment taxes.2021 December 1, Askell, A. et. al., “A General Language Assistant as a Laboratory for Alignment”, in arXiv, →DOI:
2 more examples
- We want an alignment procedure that avoids an alignment tax, because it incentivizes the use of models that are unaligned but more capable on these tasks.2022 March 4, Ouyang, L. et. al., “Training language models to follow instructions with human feedback”, in arXiv, →DOI:
- We note that instead of an alignment tax our proposal entails a safety dividend – the more rational the system the more capable and the safer it will be.2023 February 27, Kornai, A. et. al., “Safety without alignment”, in arXiv, →DOI: