跳至內容

所取所求,繄唯專意

文出維基大典
斯文取化裁之機杼英語transformer (deep learning),元元本本,發明奧義。而機杼之構材,有其主要者,插圖以示其意(英文)

所取所求,繄唯專意[註 一]西字Attention Is All You Need[]谷歌社八士合撰,西元二〇一七年發表。斯文既出,百家騰躍,終入環內,機器學習之大寶也[][]。文中倡立新法,名曰化裁英語transformer (deep learning),得爲深學之架構。其法,本乎專意之機杼英語attention mechanism,蓋推自二〇一四年德米特里·巴達納烏Dzmitry Bahdanau輩所演之術[]。斯文之作,寔人工智慧之立極[];嗣後百工競逐,開發機智華言人工智能熱潮,亦履端於斯;凡諸種靈慧,若大語言模型之屬,咸以化裁爲骨脊[][]。初,八士但欲以之善機飜華言機器翻譯纚對纚之術英語seq2seq[註 二],然亦預見其用之必廣,可及於問答之制華言問答系統[註 三],暨當世機智之能文善藝華言生成式人工智能,而得兼諸模態華言多模態學習者云[]

按樂坊云披頭士者,有名在前,曰《所取所求,繄唯愛戀英語All You Need Is Love[註 四]》;洎乎斯文,題旨之作「所取所求,繄唯專意」者,致敬華言致敬 (藝術)故也[]。至化裁之「transformer」,其名則作者雅各·烏茲科雷特Jakob Uszkoreit所定,徒以其音韻鏗鏘而好之也[]

考其初稿,曰《化裁:迭相自專其意以調處諸務》Transformers: Iterative Self-Attention and Processing for Various Tasks,中繪《變形金剛Transformers六將之像,因自號「變形金剛之隊」Team Transformer[]。及初,用以迻譯兩文;以立華言條目於《大典》,發明化裁之奧義;以解析文法英語parsing:屢有奇效。作者數人,遂謂是允稱通用之語模華言語言模型,非獨爲繙譯設也[]

斯文既出,開源發流,爲後學準。後此八士,皆谷歌,或自創會社,鑽研機智,或改就他職[一〇]。迄二〇二五年,斯文之引述,逾十七萬三千度[一一]

[溯源]
  1. 顏永京譯attention爲「專意」。
  2. Sequence,與數英語data之纚列,而前後相關聯者也。茲參學部華言學部 (清朝)編訂名詞館之審定series爲「纚列」也。
  3. 言system之富於機能華言功能 (工程)者,學部華言學部 (清朝)多以「制」「式」統而系之:若system of drainage之作「宣洩之制」,而system of locomotion之作「驛傳之式」也。茲法之。
  4. 嚴復社會通詮》也,雙稱love爲「愛戀」。

[溯源]
  1. 1 2 Vaswani氏名AshishShazeer氏名NoamParmar氏名NikiUszkoreit氏名JakobJones氏名LlionGomez氏名Aidan NKaiser氏名ŁukaszPolosukhin氏名Illia《31st Conference on Neural Information Processing Systems (NIPS)》。收「Advances in Neural Information Processing Systems」其三十,Curran Associates, Inc.刊之,arXiv: 1706.03762
  2. AI Researcher Who Helped Write Landmark Paper Is Leaving Google
  3. 'Attention is All You Need' creators look beyond Transformers for AI at Nvidia GTC: 'The world needs something better'
  4. Bahdanau氏名DzmitryCho氏名KyunghyunBengio氏名Yoshua《Neural Machine Translation by Jointly Learning to Align and Translate》。arXiv: 1409.0473
  5. Shinde氏名GitanjaliWasatkar氏名NamrataMahalle氏名ParikshitData-Centric Artificial Intelligence for Multidisciplinary Applications》第七十五頁。CRC Press刊之,二〇二五年四月十六日覽之,而原本以二〇二四年八月十九日存檔,ISBN 9781040031131
  6. Toews, Rob. Transformers Revolutionized AI. What Will Replace Them?. Forbes. 2023-09-03 [2023-12-03]. (原文存卷於2023-09-26).
  7. Murgia, Madhumita. Transformers: the Google scientists who pioneered an AI revolution. Financial Times. 2023-07-23 [2024-03-22].
  8. 1 2 Levy氏名Steven8 Google Employees Invented Modern AI. Here's the Inside Story》。收「Wired」,二〇二四年三月二十日覽之,而原本以二〇二四年三月二十日存檔,ISSN 1059-1028
  9. 1 2 Marche氏名StephenWas Linguistic A.I. Created by Accident?》。收「The New Yorker」,二〇二四年八月二十四日覽之,ISSN 0028-792X
  10. 8 Google Employees Invented Modern AI. Here's the Inside Story- Wired
  11. Meet the $4 Billion AI Superstars That Google Lost. Bloomberg. 2023-07-13 [2025-04-16]. (原文存卷於2023-08-20) 經由www.bloomberg.com.