所取所求,繄唯專意

《所取所求,繄唯專意[註 一]》(西字Attention Is All You Need)[一],谷歌社八士合撰,西元二〇一七年發表。斯文既出,百家騰躍,終入環內,機器學習之大寶也[二][三]。文中倡立新法,名曰化裁(英語:transformer (deep learning)),得爲深學之架構。其法,本乎專意之機杼(英語:attention mechanism),蓋推自二〇一四年德米特里·巴達納烏(Dzmitry Bahdanau)輩所演之術[四]。斯文之作,寔人工智慧之立極[五];嗣後百工競逐,開發機智(華言:人工智能熱潮),亦履端於斯;凡諸種靈慧,若大語言模型之屬,咸以化裁爲骨脊[六][七]。初,八士但欲以之善機飜(華言:機器翻譯)纚對纚之術(英語:seq2seq)[註 二],然亦預見其用之必廣,可及於問答之制(華言:問答系統)[註 三],暨當世機智之能文善藝(華言:生成式人工智能),而得兼諸模態(華言:多模態學習)者云[一]。
按樂坊云披頭士者,有名謳在前,曰《所取所求,繄唯愛戀(英語:All You Need Is Love)[註 四]》;洎乎斯文,題旨之作「所取所求,繄唯專意」者,致敬(華言:致敬 (藝術))故也[八]。至化裁之「transformer」,其名則作者雅各·烏茲科雷特(Jakob Uszkoreit)所定,徒以其音韻鏗鏘而好之也[九]。
考其初稿,顏曰《化裁:迭相自專其意以調處諸務》(Transformers: Iterative Self-Attention and Processing for Various Tasks),中繪《變形金剛》(Transformers)六將之像,因自號「變形金剛之隊」(Team Transformer)[八]。及初試,用以迻譯英、德兩文;以立目(華言:條目)於《大典》,發明化裁之奧義;以解析文法(英語:parsing):屢有奇效。作者數人,遂謂是允稱通用之語模(華言:語言模型),非獨爲繙譯設也[九]。
斯文既出,開源發流,爲後學準。後此八士,皆辭谷歌,或自創會社,鑽研機智,或改就他職[一〇]。迄二〇二五年,斯文之引述,逾十七萬三千度[一一]。
- 1 2 Vaswani氏(名Ashish)、Shazeer氏(名Noam)、Parmar氏(名Niki)、Uszkoreit氏(名Jakob)、Jones氏(名Llion)、Gomez氏(名Aidan N)、Kaiser氏(名Łukasz)、Polosukhin氏(名Illia)《31st Conference on Neural Information Processing Systems (NIPS)》。收「Advances in Neural Information Processing Systems」其三十,Curran Associates, Inc.刊之,arXiv: 1706.03762。
- ↑ AI Researcher Who Helped Write Landmark Paper Is Leaving Google
- ↑ 'Attention is All You Need' creators look beyond Transformers for AI at Nvidia GTC: 'The world needs something better'
- ↑ Bahdanau氏(名Dzmitry)、Cho氏(名Kyunghyun)、Bengio氏(名Yoshua)《Neural Machine Translation by Jointly Learning to Align and Translate》。arXiv: 1409.0473。
- ↑ Shinde氏(名Gitanjali)、Wasatkar氏(名Namrata)、Mahalle氏(名Parikshit)《Data-Centric Artificial Intelligence for Multidisciplinary Applications》第七十五頁。CRC Press刊之,二〇二五年四月十六日覽之,而原本以二〇二四年八月十九日存檔,ISBN 9781040031131。
- ↑ Toews, Rob. Transformers Revolutionized AI. What Will Replace Them?. Forbes. 2023-09-03 [2023-12-03]. (原文存卷於2023-09-26).
- ↑ Murgia, Madhumita. Transformers: the Google scientists who pioneered an AI revolution. Financial Times. 2023-07-23 [2024-03-22].
- 1 2 Levy氏(名Steven)《8 Google Employees Invented Modern AI. Here's the Inside Story》。收「Wired」,二〇二四年三月二十日覽之,而原本以二〇二四年三月二十日存檔,ISSN 1059-1028。
- 1 2 Marche氏(名Stephen)《Was Linguistic A.I. Created by Accident?》。收「The New Yorker」,二〇二四年八月二十四日覽之,ISSN 0028-792X。
- ↑ 8 Google Employees Invented Modern AI. Here's the Inside Story- Wired
- ↑ Meet the $4 Billion AI Superstars That Google Lost. Bloomberg. 2023-07-13 [2025-04-16]. (原文存卷於2023-08-20) – 經由www.bloomberg.com.