Advances in private training for production on-device language models
瀵瑰垬鍢夌幉鏉ヨ锛屽ス鐨勪漢鐢熼渶瑕佺暀涓€浜涚己鎲撅紝澶畬緹庣殑渚誇笉鍙綔浜虹敓浜嗐€?
鏈枃浣滆€咃細姊︽兂鏄釜鐚墠璦€榪欐瀹墮噷瑁呬慨锛屼拱浜嗕竴濂楀師鏈ㄥ鍏鳳紝鏄殑锛屾病閿欙紝灝辨槸寮犲ぇ濡堝緇忓父鎺ㄧ殑閭e嚑涓搧鐗屻€?璁茬湡锛屽湪涔拌繖浜涘鍏蜂箣鍓嶏紝涔熸槸鏈夊緢澶氱枒鎯戠殑锛屽洜涓烘湁涓ゅ紶1.5綾崇殑搴婃槸鍦ㄥ幓騫村弻鍗佷竴鐨勬椂鍊欎簲鎶樻姠璐埌鐨勶紝褰撴椂鐨勪環鏍間粎浠呭彧鏈変竴鍗冨厓鍑哄ご锛岀劧鍚庣湅浜嗗緢澶氬€煎弸閮藉湪閭i噷璿達紝榪欎袱瀹惰川閲忓浣曞浣曠殑涓嶅ソ锛屾潗璐ㄥ浣曞浣曠殑閫犲亣锛岃寰楀績閲岄兘蹇病搴曚簡銆?
鍒拌揣鍚庯紝涔熸病鏈夋媶寮€鍖呰锛屽氨鐩存帴涓㈠湪鎴戠殑浠撳簱閲屼簡锛屽績鎯籌紝鍙嶆涔熷氨閭d箞鐐歸挶锛屽疄鍦ㄤ笉琛屽氨涓嶈鍚с€?浠婂勾618鏃訛紝鍙堥檰闄嗙畫緇殑灝嗘矙鍙戙€侀妗屾銆佽尪鍑犮€佺數瑙嗘煖銆佹⒊濡嗗彴銆佷富鍗у簥銆佹蕩瀹ゆ煖絳夊叏閮ㄧ粰涔頒簡鍥炴潵锛岃繖涓椂鍊欙紝鍏跺疄瀵逛簬榪欎簺鐢靛晢鍘熸湪瀹跺叿锛屾垜宸茬粡娌¢偅涔堟媴蹇冧簡锛屽洜涓哄浜庢湪鏂欐潗璐紝宸茬粡琛ヤ範浜嗕笉灝戠煡璿嗐€?鎵€浠ヨ繖涓€綃囷紝灝卞崟鐙璿存垜榪欐瑁呬慨鎵€鐢ㄧ殑涓€浜涙湪鏂欎互鍙婅繖浜涚數鍟嗗師鏈ㄥ鍏風殑鏉愯川鍚э紝鏈枃綰補鏄粠涓漢娑堣垂瑙掑害锛屾湁涓€璿翠竴锛屼笉涓鴻皝媧楀湴锛屼篃涓嶆晠鎰忛粦璋併€?
鍏充簬瑁呬慨鐨勬湪鏂欒淇椂锛岀敤浜嗕笉灝戞湪鏂欙紝涓昏鏄墦浜嗕笉灝戠殑鏌滃瓙锛屽叏閮ㄦ槸鐜板満鏈ㄥ伐鍔犲伐鐨勶紝榪樻湁綆€鍗曠殑鍚婇《鍜岃儗鏅锛屼篃闇€瑕佷竴瀹氱殑鏈ㄦ枡锛屾墍浠ラ鍏堣璿磋淇椂甯哥敤鐨勪竴浜涙湪鏂欏惂銆?鏈ㄥ伐鏉匡紝鍦ㄦ箹鍗楀湴鍖猴紝涓昏鏄ぇ鑺澘銆佸灞傛澘銆佹寚鎺ユ澘锛屽彠澶栵紝鏈湴澶у鏁板畾鍒跺鍏鳳紝閮戒細閫夋嫨棰楃矑鏉垮拰楂樺瘑鏉褲€?
澶ц姱鏉挎槸鐢ㄦ湪鏉挎潯鎷兼帴鍚庯紝鍙岄潰瑕嗙洊涓ゅ眰鎴栬€呭灞傝兌鍚堟澘緇忚兌鍘嬪埗鎴愮殑涓€縐嶆澘鏉愶紝浠ュ墠鏈ㄥ伐涓€鑸兘鏄敤澶ц姱鏉垮埗浣滄煖浣擄紝鐒跺悗鍐嶅湪瑙佸厜闈笂璐翠竴灞傞グ闈㈡澘锛岄グ闈㈡澘涓昏鏈夋按鏇叉煶銆佺櫧姍℃湪銆侀粦鑳℃絳夛紝榪欐牱鍔犲伐鍑烘潵鐨勬煖浣擄紝琛ㄩ潰鑳藉仛鍠鋒紗鎴栬€呮摝鑹叉紗锛岃兘鏄鵑湶鍑洪グ闈㈡澘鐨勭汗鐞嗭紝鍙皳鏄墿緹庝環寤夈€?
浣嗘槸鍥犱負榪欑鍔犲伐宸ヨ壓锛岄渶瑕佺幇鎴愬仛娌規紗锛屽湪娌規紗宸ヤ環闈炲父鏄傝吹鐨勭幇鍦紝鍐嶅姞涓婃煇浜涙墍璋撶幆淇濈殑姒傚康锛岀幇鍦ㄦ湰鍦板緢澶氭湪宸ラ兘寮€濮嬮€夋嫨浣跨敤鍏嶆紗鏉褲€?鍥介檯宸ㄦ槦鎴愰緳鏇劇粡灝辨槸綾抽洩鐨勭埍鎱曡€咃紝浣嗘槸綾抽洩鏈€緇堟病鏈夊拰鎴愰緳鍦ㄤ竴璧鳳紝鑰屾槸閫夋嫨浜嗗拰灝瑰織寮鴻繘琛屼氦寰€銆?
綾抽洩鍜屽肮蹇楀己浜?984騫存寮忕‘瀹氭亱鎯咃紝灝瑰織寮哄湪1992騫村氨琚獙鍑烘偅涓婁簡榧誨捊鐧岋紝鉶界劧浜屼漢榪樻病鏈夌粨濠氾紝浣嗘槸綾抽洩涔熷浠栦笉紱諱笉寮冿紝涓€鐩撮櫔浼村肮蹇楀己鍒?010騫村幓涓栵紝鐢氳嚦鍦ㄥ肮蹇楀己鍘諱笘鍚庯紝榪樻兂瑕佸拰浠栦婦鍔炲啣濠氾紝鐢熸涓嶅純锛屽叓騫寸殑榛橀粯瀹堟姢鍙鍏舵繁鎯呫€?鐢風粓鐢熸寶鐖卞肮蹇楀己鍘諱笘鍚庯紝綾抽洩灝變竴鐩翠繚鎸佺潃鍗曡韓鐨勭姸鎬佽嚦浠婏紝浣嗘槸綾抽洩騫舵病鏈夎鎮蹭激鎵撳埌锛岃€屾槸媧誨嚭浜嗚嚜宸辯殑綺懼僵锛屽湪榪囧幓鐨勪竴騫存椂闂撮噷锛岀背闆繕緇忓父鍦ㄧぞ浜ゅ鉤鍙頒笂鍒嗕韓鑷繁榪愬姩鏃剁殑鐓х墖锛屾暣涓漢鐪嬭搗鏉ュ厖婊¢槼鍏夛紝媧誨姏鍥涘皠銆?
綾抽洩鍠滄澶х濂借韓鏉愶紝濂瑰湪鎺ュ彈濯掍綋閲囪鏃朵篃鍥炲簲榪囪繖浠朵簨锛屽ス琛ㄧず鑷繁涓嶆槸涓轟簡鐐€€鑷繁鐨勫ソ韜潗锛岃€屾槸涓轟簡榧撳姳鏇村浜哄弬涓庤繍鍔紝濂矽繕閫忛湶鍑鴻嚜宸變粠鍘誨勾灝卞凡緇忓紑濮嬩簡姣忓ぉ鍋氳繍鍔ㄦ潵緇存寔鑷繁鐨勮壇濂戒綋鎬侊紝騫惰〃紺鴻繍鍔ㄦ墠鏄漢鐢熸渶鍊煎緱鎶曡祫鐨勪簨鎯呫€?闄や簡涓€鐩村潥鎸佽繍鍔ㄤ箣澶栵紝綾抽洩鍦ㄤ簨涓氫笂涔熷苟娌℃湁鏀炬澗锛岃€屾槸渚濇棫淇濇寔楂樹駭鐘舵€併€?