2024人工智能風(fēng)險(xiǎn)國(guó)際科學(xué)評(píng)估的未來(lái)

RESEARCHMEMOAUGUST2024

TheFutureofInternationalScientificAssessmentsof

AI’sRisks

Authors:HadrienPouget*,ClaireDennis*,JonBateman,RobertF.Trager,Renan

Araujo,HaydnBelfield,BelindaCleeland,MalouEstier,GideonFuterman,OliverGuest,CarlosIgnacioGutierrez,VishnuKannan,CaseyMahoney,MatthijsMaas,Charles

Martinet,JakobM?kander,KwanYeeNg,Seánóhéigeartaigh,AidanPeppin,KonradSeifert,ScottSinger,MaximeStauffer,CalebWithers,andMartaZiosi

*Equalcontribution.Nameorderrandomised.Pleaseciteas:'Pouget,H.,Dennis,C.,etal.(2024)'

Inpartnershipwith

CARNEGIE

ENDOWMENTFOR

INTERNATIONALPEACE

Contents

ExecutiveSummary1

Introduction5

ATaskTooAmbitiousforAnyOneOrganization7

Recommendations9

Conclusion17

Appendix19

AbouttheAuthors23

Notes27

CarnegieEndowmentforInternationalPeace29

OxfordMartinAIGovernanceInitiative31

1

ExecutiveSummary

Managingtherisksofarti?cialintelligence(AI)willrequireinternationalcoordination

amongmanyactorswithdiferentinterests,values,andperceptions.Experiencewithotherglobalchallenges,likeclimatechange,suggeststhatdevelopingashared,science-based

pictureofrealityisanimportant?rststeptowardcollectiveaction.Inthisspirit,lastyeartheUKgovernmentledtwenty-eightcountriesandtheEuropeanUnion(EU)inlaunchingthe

InternationalScienti∮cReportontheSafetyofAdvancedAI.

meUK-ledreporthasaccomplishedagreatdealinashorttime,butitwasdesignedwithanarrowscope,limitedsetofstakeholders,andshortinitialmandatethat’snownearingitsend.Meanwhile,theUnitedNations(UN)isnowmovingtowardestablishing

itsown

reportprocess

,thoughkeyparametersremainundecided.Andahodgepodgeofotherenti-ties—includingtheOrganisationforEconomicCo-operationandDevelopment(OECD),theemergingnetworkofnationalAISafetyInstitutes(AISIs),andgroupingsofscientistsaroundtheworld—areweighingtheirownpotentialcontributionstowardglobalunder-standingofAI.

Howcanalltheseactorsworktogethertowardthecommongoalofinternationalscienti?cagreementonAI’srisks?merehasbeensurprisinglylittlepublicdiscussionofthisquestion,evenasgovernmentsandinternationalbodiesengageinquietdiplomacy.Moreover,thedif-?cultyofthechallengeisnotalwaysfullyacknowledged.Comparedtoclimatechange,forexample,AI’simpactsaremoredi伍culttomeasureandpredict,andmoredeeplyentangledingeopoliticaltensionsandnationalstrategicinterests.

Todiscussthewayforward,OxfordMartinSchool’sAIGovernanceInstituteandtheCarnegieEndowmentforInternationalPeacebroughttogetheragroupofexpertsattheintersectionofAIandinternationalrelationsinJuly.Drawingfromthatdiscussion,sixmajorideasemerged:

?Nosingleinstitutionorprocesscanleadtheworldtowardscienti?cagreementonAI’srisks.merearetoomanyconAictingrequirementstoaddresswithinasingleframeworkorinstitution.Globalpoliticalbuy-independsonincludingabroad

rangeofstakeholders,yetgreaterinclusivityreducesspeedandclarityofcommonpurpose.Appealingtoallglobalaudienceswouldrequirecoveringmanytopics,andcouldcomeatthecostofcoherence.Scienti?crigordemandsanemphasisonpeer-reviewedresearch,yetthisrulesoutthemostcurrentproprietaryinformationheldbyindustryleadersinAIdevelopment.Becausenooneefortcansatisfyallthesecompetingneeds,multipleefortsshouldworkincomplementaryfashion.

?meUNshouldconsiderleaningintoitscomparativeadvantagesbylaunchingaprocesstoproduceperiodicscienti?creportswithdeepinvolvementfrommemberstates.SimilarlytotheIntergovernmentalPanelonClimateChange(IPCC),thisapproachcanhelpscienti?cconclusionsachievepoliticallegitimacy,andcannur-turepolicymakers’relationshipsandwill-to-act.mereportscouldbeproducedoveracyclelastingseveralyearsandcoverabroadrangeofAI-relatedissues,bringing

togetherandaddressingtheprioritiesofavarietyofglobalstakeholders.Incontrast,apurelytechnical,scientist-ledprocessunderUNauspicescouldpotentiallydilutethecontentonAIriskswhilealsofailingtoreapthelegitimatingbene?tsofthe

UN’suniversaliststructure.

?Aseparateinternationalbodyshouldcontinueproducingannualassessments

thatnarrowlyfocusontherisksof“advanced”1AIsystems,primarilyledby

independentscientists.merapidtechnologicalchange,potentialscaleofimpacts,andintensescienti?cchallengesofthistopiccallforadedicatedprocesswhichcanoperatemorequicklyandwithmoretechnicaldepththantheUNprocess.ItwouldoperatesimilarlytotheUK-ledreport,butwithgreaterglobalinclusion,drawingdatafromawiderrangeofsourcesandwithinapermanentinstitutionalhome.

meUNcouldtakethison,butattemptingtoleadboththisreportandtheabove

reportunderasingleorganizationriskscompromisingthisreport’sspeed,focus,andindependence.

?mereareatleastthreeplausible,ifimperfectcandidatestohostthereportdedi-catedtorisksfromadvancedAI.menetworkofAISIsisalogicalsuccessortotheUK-ledefort,butitfacesinstitutionaluncertainties.meOECDhasastrongtrackrecordofsimilarwork,thoughitremainssomewhatexclusive.meInternational

ScienceCouncilbringslessgeopoliticalbaggagebuthasweakerfundingstructures.

2|TheFutureofInternationalScienti?cAssessmentsofAI’sRisks

3

Regardlessofwholeads,alloftheseorganizations—andothers—shouldbeactivelyincorporatedintoagrowing,globalpublicconversationonthescienceofadvancedAIrisks.

?metworeportsshouldbecarefullycoordinatedtoenhancetheircomplemen-

taritywithoutcompromisingtheirdistinctadvantages.SomecoordinationwouldenabletheUNtodrawontheindependentreport’stechnicaldepthwhilehelpingitgainpoliticallegitimacyandinAuence.However,excessiveentanglementcouldsloworcompromisetheindependentreportanderodetheinclusivityoftheUNprocess.Promisingmechanismsincludememorandaofunderstanding,mutualmembershiporobserverstatus,jointlyrunningevents,presentingonintersectingareasofwork,andsharingoverlappingadvisors,experts,orstaf.

?ItmaybenecessarytocontinuethecurrentUK-ledprocessuntilotherprocessesbecomeestablished.Anynewprocesswilltaketimetoachievestakeholderbuy-in,negotiatekeyparameters,hirestaf,buildworkingprocesses,andproduceoutputs.memomentumandsuccessoftheUK-ledprocessshouldnotbesquanderedafterthe?rsteditionispresentedatFrance’sAIActionSummitinFebruary.

5

Introduction

InternationalcoordinationwillbeanecessarypartofaddressingAI’sglobalimpactsandefectivecoordinationdemandsashared,scienti?callyrigorousunderstandingofAIrisks.Whileitdoesnotguaranteeinternationalcooperation,sharedscienti?cunderstandinghasbeenanecessarypreconditionforprogressinothergloballysigni?cantpolicydomainslikeclimatechange,2biodiversityandecosystems,3andradiationrisks.4

However,severalchallengesmakethistaskdeeplycomplexinthecontextofAI.Areasofscienti?cagreementanddisagreementareinconstantAuxasthetechnologicalfrontier

evolvesquicklyandAIsystemsaredeployedinnewways.Informationaboutsystems’

currentcapabilitiesandimpactsisnotsystematicallycollectedorpublished.Moreover,

muchofthiscrucialdataisproprietaryandcloselyguardedbyprivateAIcompaniesduetocompetitivecommercialinterests.AttemptingtomakepredictionsabouttheevolutionofthetechnologyisevenmorefraughtduetotheunpredictablenatureofAIadvancements.InternationalefortswillberequiredtosystematicallyandfrequentlyassesstheimpactsofAI.ToensurethisassessmentreAectsthedistinctcontextsofcountriesaroundtheworld,itrequiresaccuracyandlegitimacyneededtoserveasaspringboardforinternationalaction.

InJuly,theOxfordMartinSchool’sAIGovernanceInitiativeandCarnegieEndowment

forInternationalPeaceco-hostedanexpertworkshop—heldundertheChathamHouserule—toexploreoptionsforthefutureofinternationalscienti?cassessmentsofAI’srisksincludingefortsledbytheUKandtheUN.Basedoninsightsfromthatworkshop,thispaperexploresthefullpotentialambitionofinternationaleforts,recognizestensionsbe-tweendiferentgoals,andmakesthreerecommendationsforbalancingtheneedfortimely,

scienti?callygroundedupdateswithglobalinclusivenessandinternationallegitimacy.Italsotakesparticularnoteoftheurgencyanddi伍cultyofassessingemergingrisksfromthemostadvancedAIsystems,whichhavedevelopedrapidlyinrecentyears.

TheInternationalScienti?cReportontheSafetyofAdvancedAI

Buildingashared,science-basedinternationalunderstandingofAIrisksisnotanentirely

newgoal.miswasonemotivationforthelaunchin2018of

GlobalPartnershiponAI

(GPAI),abodycomprisedoftwenty-eightcountriesandtheEU.GPAIhascarriedoutre-searchonarangeofAIissues,aimingtobridgetheoryandpractice.5However,thereleaseofChatGPTinlate2022creatednewurgencyandbroughtparticular

attention

totheimpactsofadvancedAIsystems,providingmomentumforadedicatedefort.

AttheAISafetySummitheldinBletchleyParklastNovember,theUKgovernment,withsupportfrom

twenty-eightcountriesandtheEuropeanUnion

,

commissioned

a“state

ofthescience”reportonrisksfromadvancedAIsystems.LedbygloballyrecognizedAI

expertProfessorYoshuaBengio,thisinitiativeaimedto“facilitateasharedscience-based

understandingoftherisksassociatedwithfrontierAI.”Aninterimversionofthereport,the

InternationalScienti∮cReportontheSafetyofAdvancedAI

,waspresentedattheAISeoul

SummitinMay2024,andthe?nalversionisexpectedbeforetheFrenchgovernment’sAIActionSummitinFebruary2025.mereportprimarilysynthesizesacademicresearchonAIsafety,focusingonthreekeyareas:1)theexpandingcapabilitiesofadvancedAIsystems,whichareaprimarydriverofrisk;2)thecurrentstateoftechnicalabilitiestoevaluate

modelsandmitigaterisks;and3)speci?criskcategories,includingmalicioususeandsystemicimpacts.

meinterimreportrepresents,sofar,themostsigni?cantsteptowardaglobalunderstandingofrisksfromadvancedAI.Itprovidedasoberyetprudentoverviewofkeyrisks,acknowl-edgingmajorareasofuncertaintyanddebate—nosmallfeat,giventhelimitedtimeand

resourcesavailable.Importantly,thereportprocessdrewuponadiversegroupofwriters,

representingvariousinstitutions,geographicregions,andareasofexpertisewithinAIand

related?elds.Aninternational

expertadvisorypanel

composedofrepresentativesfromthirtynations,aswellasfromtheEUandtheUN,hadtheopportunitytoprovidefeedback(anddissentingviews)duringthewritingprocess.Anothergroupofsenioradvisorsfromaca-

demia,civilsociety,industry,andgovernmentbodiesalsocontributedtotheprocess.

However,theprocesshadseverallimitations.First,thescopewaskeptintentionallynarrow,focusingsolelyon“advanced”and“general-purpose”AI.mesetypesofsystemscomewithdistinctrisksthatmeritspecialattention,buttheyneverthelesscompriseonlyasmallsliceofAI—omittingtoolslikerecommendationalgorithmsandfacialrecognitionsystemsthatarealreadyhavinglargeimpactsonsocieties.Second,thereportfocusedonlyonrisks,evenasmanycountriesandcompaniescarejustasmuchormoreaboutthepotentialbene?tsofAI.

6|TheFutureofInternationalScienti?cAssessmentsofAI’sRisks

7

meIrishrepresentativetotheexpertpanelpublishedabriefdissentthat“notedconcernthatthegeneraltoneofthereportisexcessivelynegative.”mird,althoughthereportincorpo-

ratedarangeofinternationalperspectives,theprocesswasnotwhollyinclusive.Onlyafewdozencountrieswereinvolved,largelyhigh-income,andtheUKgovernmentunilaterallyestablished

theprocedures

bywhichthereportiscurrentlybeingproduced,grantingfulleditorialcontroltothechair.Fourth,thereportreliedsolelyon“high-quality”publishedsources,withnomechanismtoincorporateclassi?edorproprietarydata,ortoreceiveandconsiderviewsfromthepublic.

Limitationsofthissortrepresentinevitablecompromises,particularlyforsuchanewand

urgentendeavor.Infact,thereport’sone-yearinitialmandateisanimplicitacknowledgmentthatfutureiterationsmightneedtolookdiferent.Ultimately,theworldwillneedamore

enduringandinstitutionalizedsetofprocessestopromotecommonscienti?cunderstandingsofAIrisks.

AnEmergingUNProcess

Recentdraftsofthe

GlobalDigitalCompact

(GDC)haveproposedtheestablishmentofanInternationalScienti?cPanel(ISP)onAIwithintheUNthatwouldbetaskedwith

producingannualscienti?cassessmentsofAIrisks.Whilethecompactisstillundernegoti-ation,earlysignssuggesttheproposalfortheInternationalScienti?cPanelwillbeincludedwhenthecompactis?nalizedandadoptedbymemberstatesfor

SummitoftheFuture

inSeptember2024.misisinlinewithasimilarrecommendationfromthe

interimreport

oftheUN’sHigh-LevelAdvisoryBodyonAI.

AUNprocesswouldbeanincrediblysigni?cantdevelopmentinthisspace,giventheUN’suniquepositioninworldpoliticsanditswidemembership.mekeyquestionforthispaperisthereforenotwhethertheUNshoulddosomething,butwhatitshoulddo,andhowotheractorscancomplementandenhanceitswork.Forexample,theUK-ledreport,oranydirectsuccessor,wouldneedtoberede?nedwithrespecttotheUN’sefort.

ATaskTooAmbitiousforAnyOneOrganization

Anidealapproachwouldentailanannual(ormorefrequent)scienti?callyrigorousassess-mentthatcoversthefullspectrumofpertinentissues.Itwoulddrawuponcurrentdataanddiverseglobalperspectives,bringingtogetheracademics,policymakers,civilsociety,and

membersofindustry—ultimatelyachievingbuy-infromglobalpolicymakers.Itwouldaim

tobepolicy-relevant,inthesensethatitscontentspeaksdirectlytoandisrelieduponbypolicymakersonthemajorissuestheyface.Yetitshouldremainpolicy-neutral,notpre-

scribingactionsorsolutions,toretainscienti?ccredibilityandavoidbecomingentangledinglobalpolitics.6

Individually,thesearechallenginggoals.Collectively,theyaredaunting.Inherenttensionsamongthemwouldmakeitnearlyimpossibleforanysingleorganizationtoaccomplishalltheseambitionsinasingleprocess.

First,thebasicdecisionofscopepresentsanimmediatedilemma.KeepingthescopenarrowcouldrepresentanimplicitprioritizationofadvancedAIrisksontheglobalstage,frustratingcountrieswhoaremoreconcernedwiththeeconomicandpoliticalconsequencesoffailingtotakeadvantageandownershipofthetechnology.Suchcountriesmaycomprisemuchoftheworld—includinglow-incomecountries,butalsomajorpowerslike

China

and

France

whoaspiretogrowtheirAIecosystems.Withoutthesupportofthesecountries,thereport’s?ndingsmaynottranslateintorobustglobalaction.However,wideningthescopeofany

reporttohelpbuildbroaderbuy-increatesproblemsofitsown.Itwouldmakeithardertomovequicklyandwouldofermoresurfaceareafordisagreementtoariseandderailtheprocess—forexample,ifcountriesliketheUnitedKingdomandtheUnitedStatesbecomeconcernedthatdiscussionsofriskarebeingdiluted.

mentherearethetechnicalchallenges.StayingabreastofthelatestdevelopmentsintheAIworldwouldprovedi伍cultevenforanassessmentnarrowlyfocusedonrisksfromadvancedAIsystems,letaloneAIingeneral.Paintinganup-to-datepictureoftheriskswouldinvolveasigni?canteforttodrawonawiderrangeofsourcesthantraditionalscience-basedo伍cialreports,whichhavetypicallyreliedonpeer-reviewedliteratureanddatafrommember

states.7metraditionalapproachworkslesswellforstudyingadvancedAIbecausethescien-ti?clandscapeissofast-movingthatrigorous,groundbreakingresearchisoftenpublishedoutsideofthepeerreviewprocesstooptimizeforimpact.meUK-ledreport

explains

that“notallsourcesusedforthisreportarepeer-reviewed”forexactlythisreason.meprivate

sector’sespeciallylargeroleinAI’sscienti?ccommunityfurthercomplicatesthings,asmuchofthemostup-to-datedataonthefrontierofAIcapabilitiesandontheuseandimpactsofthesystemsisproprietary.8

Keepingthereportcurrentwouldthereforerequirepullingondiversesourcesofdata,

notablyfromacademiaandtheprivatesector,butalsogovernmentsandcivilsocietyaroundtheworld.Doingsowouldentailnavigatingcomplexrelationshipswithcompanieswhere

conAictsofinterest,legalconcerns,andcompetitioncouldcolorthedatatheychooseto

share.Itwillalsorequirerigorouslyvettingthesourcesofdataandinformationthatareusedingeneral,ifacademicpeer-reviewcannotbefullyreliedupon,anditcouldincludedrawingoninputsinmanydiferentlanguagesandformats,posingapracticalchallenge.Taken

togetherthesechallengeswouldmaketrackingtherapidlyevolvingscienti?cunderstandingoftherisksandwritingyearlyscienti?creportsanintensiveprocess.

8|TheFutureofInternationalScienti?cAssessmentsofAI’sRisks

9

Addressingthesetechnicalchallengesmightrequireempoweringasizablegroupofhighlycompetentscientiststoactindependently.However,thatapproachcouldrunatcross-pur-

poseswiththeultimategoalofhavingglobalpolicymakersacknowledgetheassessment’s

?ndingsasalegitimatebasisforcoordinatinginternationalaction.Scienti?crigor,byitself,isinsu伍cienttoachievepoliticalbuy-inandperceivedlegitimacy—theremustalsobemean-ingfulrepresentationofthevarietyofexistingperspectives.Involvingpolicymakersinthe

draftingprocessisapowerfultoolforbuildingengagementandacceptanceofthe?ndings,butofcourse,thiscanslowtheprocessandpotentiallydilutethe?ndingsaspoliticalinter-estscomeintoplay.misisaparticularriskinareasofgreatuncertainty,suchasassigninglikelihoodstofuturescenarios.meIPCCprocess,forinstance,hasbeen

accused

ofmakingoverly

optimistic

predictionsaboutclimateimpactsinthepursuitofpoliticalcompromise.

Itisdi伍culttoenvisionasingleorganizationorinstitution,includingtheUN,thatcouldaccomplishthesecompetingtasksandresolvetheseinherenttensions.However,twoormoreentitiesworkingtogether,orinparallel,couldpotentiallycomplementeachotherbyleaningintotheirrespectivecomparativeadvantages.menextsectionsexplorewaysofdoingthis.

Recommendations

Recommendation1:TheUNProcessShouldFocusonMeaningfullyEngagingMemberStates

InsettinguptheprocessandmandatefortheInternationalScienti?cPanelonAI,we

recommendtheUNleanintoitscomparativeadvantageofbringingtogetherglobal

policymakers.Itshouldaimtoboostglobalbuy-inthroughtheco-creationofitsreport

betweenscientistsandpolicymakersinamannersimilartotheIPCC,ratherthanhavingacompletelyindependentscienti?cprocess.Inadditiontothe?ndingsofthereport,thecon-sensus-buildingprocessispoliticallyvaluableinandofitself,particularlygiventhatactorsinvolvedinthereportwillintersectwithotherinternationalfora,domesticpolicyprocesses,andAIdevelopment.Ofcourse,amemberstate–drivenprocesswillbeslowandlessscientif-icallyindependent.Butthesegapscanbeaddressedwithaseparate,complementaryprocesspurpose-builtforthetask(seeRecommendation2).TryingtoaccomplisheverygoalwithasingleUNprocessrisksamuddledoutcomethatfallsshortonallscores.

Tobringmemberstatestogether,webelievethescopeofissuestackledbytheUNprocessshouldbebroaderthanrisksfromadvancedAIsystems,andcouldincludeawiderangeofAIrisksandbene?ts.Whatpreciselyiscoveredwillultimatelybeacompromisebetweenmemberstates,althoughwewouldsuggestfocusingontheissuesthatmostrequireinterna-tionalcoordination9andbeingmindfuloftopicsalreadycoveredbyinternationalprocesses

toavoidcompetingorduplicating.meUNcouldsplitthereportintodiferentworking

groupstoensurethattheprocessremainsrelevantandvaluabletoallmemberstates,regard-lessoftheirlevelofAIdevelopment.mesediferentgroupsshouldformexternalpartner-shipswithorganizationsbestsuitedforeachissue.

DrawingInspirationFromExistingOrganizations

WhilewerecommendaimingforamodelliketheIPCC,itisworthrecognizingthat

inpractice,thismaybetooambitious.Similarefortshavesince

failed

togainthesame

tractionastheIPCC,theIPCCitselfhas

struggled

withgrowingpoliticization,andinter-governmentalinstitutionsingeneralhavebeenincreasingly

gridlocked

.Indesigningthedetailsofthereport-writingprocess,theUNcandrawinspirationfromitsmultitudeofexistingandassociatedscienti?cbodies.WebrieAyoutlinetheIPCCaswellasacoupleofothermodelsforhowmemberstatescanbeengaged.

IPCCModel

me

IPCC

isperhapsthemostfamousexampleofsuchaprocess.Itsreportsfeaturea

summaryforpolicymakers(SPM)whichrequires

line-by-line

approvalbygovernmentrep-resentativesfrom195memberstates.Appointedscientistscollaborateinworkinggroupsonthematicareas(forexample,physicalsciencebases,impactsandadaptation,andmitigation),relyingonpeer-reviewedacademicliteraturetoproducethedeeperscienti?creportupon

whichtheSPMsarebased.mehighlevelofpoliticalbuy-infacilitatesdirectincorporationof?ndingsintonationalandinternationalpolicies,butthewholeprocesstypicallytakes?vetosevenyearsandrequiressigni?cantinvestmentandexpertisefromparticipatingstates.

IPBESModel

Anotherpossiblemodelisthe

IntergovernmentalScience-PolicyPlatformonBiodiversity

andEcosystemServices

(IPBES),whichisstructuredsimilarlytotheIPCCbutprovides

moreAexibilityandopportunityfordiverseinput.10Memberstatesparticipateintheinitialscopingprocess,wheretheobjectives,scope,andoutlineoftheassessmentaredetermined.WhiletheSPMsarealsoapprovedbymemberstates,theprocessistypicallylessdetailed

thantheIPCC’sline-by-linescrutiny.Workinggroupsareformedanddissolvedaccordingtodemand,ratherthanbeing?xed,anddrawfromdiversesourcesincludingindigenousandlocalknowledge(ILK)and“greyliterature”suchasgovernmentreports,nonpro?tpubli-

cations,andothernon-peer-reviewedsourceswhentheyaredeemedrelevantandcredible.IPBESalsomoreproactivelyandformallyseeksmultistakeholderinputthroughoutits

processesthantheIPCC.However,thisincreasedAexibilitycameatthecostofconsistencyandpolicyrelevance,anda

2019review

suggesteditneededtodomoretoengagestrategicstakeholdersandfeedintopolicy.

10|TheFutureofInternationalScienti?cAssessmentsofAI’sRisks

11

UNSCEARModel

Finally,the

UnitedNationsScienti?cCommitteeontheEfectsofAtomicRadiation

(UNSCEAR)

representsathirdmodel,characterizedbylightertouchmember-statein-

volvementwhichwebelievewouldworklesswellforAI.UNSCEARstudiesthelevelsandefectsofexposuretoionizingradiationthroughacommitteeofscientistsappointedbytheUNGeneralAssembly(UNGA)andsupportedbyasecretariat.Governmentsprovidedata,commissionreports,andreviewdrafts,butdonotapprovethem.Nonetheless,thelegitima-cyofUNSCEAR’sreportsisreinforcedbyUNGAresolutionsacknowledgingthereports,andUNSCEARisuniqueamongtheorganizationswementioninreportingdirectlytotheUNGA.However,thissetupisinpartpossiblebecauseUNSCEARdealswitharelativelystable?eldofradiationscience,

explicitly

staysawayfromeconomicbene?tsofthetechnol-ogyorsafetystandards,andhasonlythirty-onecountriesonits

scienti?ccommittee

—incontrastwiththerapidlyevolving?eldofAI,whichisdeeplyenmeshedwitheconomicandsecurityissuesandrequiresglobalattention.

Recommendation2:AnAnnual,IndependentScienti?cReportonAdvancedAIRisksShouldBeMaintained

AUNprocessalongthelinesdescribedinRecommendation1wouldbeahugelyambitiousproject.Evenso,itwouldnotbefullysu伍cientbyitself.AdeliberateandinclusiveUN

processthataddressesabreadthofAIrisksandbene?tswould,inparticular,struggletogiveadequatefocustorisksfromadvancedAI—anespeciallychallengingsubject.AUNreportwouldpotentiallytaketoolonggiventhepaceofscienti?cadvancement,betoovulnerabletopoliticization,andbetoobroadtotackletheincrediblydemandingscienti?ctasks.We

thereforeproposeanotherprocess,intendedtofollowandbuildontheUK-ledreport,whichwouldcomplementandsupporttheUNwork.

LikethecurrentUK-ledprocess,thenewprocessonrisksfromadvancedAIwouldproducereportsatleastyearly,witheditorialcontrolgiventoaninternationalgroupofindependentacademics.Policymakerinputwouldbelimited—forexample,endorsingthescopeofques-tionstobeconsidered.Ahighlevelofindependenceandtechnicalcompetencewouldenablewriterstomakeconnectionswithawiderangeofstakeholders,includingindustry,andvetdiferentsourcesofinform