




版權(quán)說明:本文檔由用戶提供并上傳,收益歸屬內(nèi)容提供方,若內(nèi)容存在侵權(quán),請進(jìn)行舉報或認(rèn)領(lǐng)
文檔簡介
PrivacyAttacks
&
Defenses姜育剛,馬興軍,吳祖煊SalvadorDalí,“ThePersistenceofMemory,”1931Recap:
week
8Data
Extraction
Attack
&
DefenseModel
Stealing
AttackFuture
ResearchThis
WeekMembership
Inference
AttackDifferential
PrivacyMembership
Inference
AttackDifferential
PrivacyMembership
Inference
AttackMembership
Inference
Attack推理一個輸入樣本是否存在于訓(xùn)練數(shù)據(jù)集中Shokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."
S&P,2017.存在?Privacy
and
Ethical
Problems
MIA
could
cause
the
following
harms:Leak
private
info:
someone
has
been
to
some
place
or
having
an
unspeakable
illness
Expose
info
about
the
training
dataMIA
sensitivity
also
indicates
data
leakage
riskAn
Early
WorkHomer,Nils,etal."ResolvingindividualscontributingtraceamountsofDNAtohighlycomplexmixturesusinghigh-densitySNPgenotypingmicroarrays."
PLoSgenetics
4.8(2008):e1000167.判斷個人基因是否出現(xiàn)在一個復(fù)雜的混合基因里可用于調(diào)查取證MIA:The
Most
Well-known
WorkShokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."
S&P,2017.0Black-box
attack
pipelineNeeds
probability
vectorMIA:The
Most
Well-known
WorkShokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."
S&P,2017.Train
k
shadow
models
on
disjoint
datasetsSample
a
number
of
subsets
from
DTrain
a
model
on
each
of
the
subsetTake
one
model
as
the
targetTake
the
rest
models
as
shadow
modelsMIA:The
Most
Well-known
WorkShokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."
S&P,2017.Different
ways
to
get
the
training
data:Random
SynthesisData
synthesisPhase
1:
searching
for
high
confidence
data
points
in
the
data
spacePhase
2:
samplesyntheticdatafromthesepointsRepeat
the
above
for
each
class
cPhase
1:每次只改變已找到的高置信度樣本的k個特征MIA:The
Most
Well-known
WorkShokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."
S&P,2017.Statistics-basedsynthesisPrior
knowledge:The
marginal
distribution
w.r.t.
each
classPhase
1:
sample
according
to
the
statisticsMIA:The
Most
Well-known
WorkShokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."
S&P,2017.We
could
also
assume
the
attacker
can
access
NoisyRealdata:
real
but
noisyVery
similar
to
the
real
datasetBut
with
a
few
features
(10%
or
20%)
are
randomly
resetMIA:The
Most
Well-known
WorkShokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."
S&P,2017.Finally:
training
the
inference
model”in”:
in
the
training
set”out”:
:
in
the
test
setTrain
the
inference
model
with
dataset:
(prob1,
”in”),
(prob2,
”in”),
(prob3,
”out”)
(prob4,
”out”)MIA:The
Most
Well-known
WorkShokri,Reza,etal."Membershipinferenceattacksagainstmachinelearningmodels."
S&P,2017.How
well
can
MIA
perform?數(shù)據(jù)集:CIFAR-10、CIFAR-100、Purchases、Locations、Texashospitalstays、MNIST、UCIAdult(CensusIncome).White-box
MIANasr
et
al.“Comprehensiveprivacyanalysisofdeeplearning:Passiveandactivewhite-boxinferenceattacksagainstcentralizedandfederatedlearning.”
S&P,2019.
Hu,Hongsheng,etal."Membershipinferenceattacksonmachinelearning:Asurvey."
ACMComputingSurveys(CSUR)
54.11s(2022):1-37.White-boxvs
Black-boxWhite-box
MIANasr
et
al."Comprehensiveprivacyanalysisofdeeplearning:Passiveandactivewhite-boxinferenceattacksagainstcentralizedandfederatedlearning."
S&P,2019.抽取特征:概率、中間層激活、梯度無監(jiān)督設(shè)置下的重構(gòu)損失推理結(jié)果Limitations
of
MIAConstructing
shadow
modelsAssuming
access
to
some
data
or
prior
knowledgeOverfitting
is
a
mustLimited
to
classification
modelsLimited
to
small
modelsAddressing
Limitations
of
MIASalemetal."ML-Leaks:ModelandDataIndependentMembershipInferenceAttacksandDefensesonMachineLearningModels."
NDSS,2019.Model
and
Data
Independent
MIAAddressing
Limitations
of
MIALong,Yunhui,etal."Apragmaticapproachtomembershipinferencesonmachinelearningmodels."
EuroS&P,2020.Attacking
non-overfitting
DNNsFocusing
on
minimizingfalsepositives目標(biāo)問題:樣本A/B在哪個模型的訓(xùn)練數(shù)據(jù)里?Addressing
Limitations
of
MIALeino
&
Fredrikson."StolenMemories:LeveragingModelMemorizationforCalibratedWhite-BoxMembershipInference."
USENIXSecurity,2020.More
practical
white-box
threat
modelThe
adversary
only
knows
the
model
but
not
the
data
distribution利用詭異的獨家記憶進(jìn)行成員推理Training
imagesInternal
explanations
Pink
background
explanation
of
Tony
BlairAddressing
Limitations
of
MIAHayes,Jamie,etal."Logan:Membershipinferenceattacksagainstgenerativemodels."
arXivpreprintarXiv:1705.07663
(2017).Extension
to
generative
models充分利用判別器的判別能力:高置信度的大概率來自原始訓(xùn)練數(shù)據(jù)集Metric-guided
MIAYeom,Samuel,etal.“Privacyriskinmachinelearning:Analyzingtheconnectiontooverfitting.”
CSF,
2018.
Salemetal."ML-Leaks:ModelandDataIndependentMembershipInferenceAttacksandDefensesonMachineLearningModels."
NDSS,2019.Metric
based
Anomaly
detection預(yù)測正確性:預(yù)測正確的就是成員預(yù)測損失:高于訓(xùn)練樣本平均損失的是成員預(yù)測置信度:有概率接近1的是成員預(yù)測熵:低概率熵的是成員修正預(yù)測熵:不同類別區(qū)別考慮A
Summary
of
Existing
MIAsUsed
DatasetsImage:CIFAR-10,CIFAR-100,MNIST,Fashion-MNIST,YaleFace,ChestX-ray8,SVHN,CelebA,ImageNetTabulate:Adult,Foursquare,Purchase-100,Texas100,Location,etc.Audio:LibriSpeech,TIMIT,TED
Text:Weibo,TweetEmoInt,SATED,Dislogs,Redditcomments,Cora,
Pubmed,CitesserHu,Hongsheng,etal.“Membershipinferenceattacksonmachinelearning:A
survey.”
ACMComputingSurveys,
2022.A
Summary
of
Existing
MIAsTargetmodels:Onimage:Multi-layerCNN+1or2FC(>5papersused2-4layersCNN)Alexnet,ResNet18,ResNet50,VGG16,VGG19,DenseNet121,Efficient-netv2,EfficientNetB0GAN:InfoGAN,PGGAN,WGANGP,DCGAN,MEDGAN,andVAEGANOntabulate
data:FConlymodelsOntext:Multi-layerCNN,multi-layerRNN/LSTM,
transformers(e.g.,BERT,GPT-2)Onaudio:Hybridsystem:HMM-DNNmodelEnd-to-end:Multi-layerLSTM/RNN/GRUMLaaS(Online):GooglePredictionAPI,AmazonMLMembership
Inference
AttackDifferential
PrivacyDifferential
PrivacyFinite
Difference
and
Derivativeh
tends
to
be
small(zero)通過函數(shù)在某一點隨微小擾動的變化可以估計在這一點的梯度如果對數(shù)據(jù)集進(jìn)行微小擾動呢?Differential
PrivacyFinite
Difference
->
Differential
Privacy數(shù)據(jù)集的微小變化會導(dǎo)致多大的算法輸出變化?
函數(shù)
輸入值
Differential
Privacy
數(shù)據(jù)集的微小變化會導(dǎo)致多大的算法輸出變化?
Differential
Privacy
Dwork,Cynthia."Differentialprivacy:Asurveyofresults."
ICTAMC,Heidelberg,2008.Properties
of
DPMcSherry,FrankD.“Privacyintegratedqueries:anextensibleplatformforprivacy-preservingdataanalysis.”
ACM
SIGMOD,2009.How
to
Obtain
a
Differentially
Private
Model?思考:如何讓自己的聲音不被發(fā)現(xiàn)??Measuring
SensitivityNissimandAdam.“Smoothsensitivityandsamplinginprivatedataanalysis.”
STOC,2007.Noise
Models幾種噪聲添加機制拉普拉斯機制(Laplacian)高斯機制(Gaussian)指數(shù)機制:離散->
概率;確定->不確定The
Laplace
Mechanism拉普拉斯機制(Laplace
Mechanism)
SaTML2023-GautamKamath-AnIntroductiontoDifferentialPrivacyThe
Laplace
Mechanism拉普拉斯機制(Laplace
Mechanism)
SaTML2023-GautamKamath-AnIntroductiontoDifferentialPrivacyThe
Laplace
Mechanism
SaTML2023-GautamKamath-AnIntroductiontoDifferentialPrivacyLaplace
vs.
Gaussian
SaTML2023-GautamKamath-AnIntroductiontoDifferentialPrivacyDP
+
Deep
Learning問題:在哪里添加噪聲?輸入空間模型空間輸出空間輸入空間DP差分隱私預(yù)處理訓(xùn)練數(shù)據(jù)dp-GAN
pipelineZhang
et
al.“Differentiallyprivatereleasingviadeepgenerativemodel(technicalreport).”
arXiv:1801.01594
(2018).輸入空間DP隨機平滑Randomized
Smoothing隨機平滑:可驗證對抗防御Cohen,Jeremy,ElanRosenfeld,andZicoKolter."Certifiedadversarialrobustnessviarandomizedsmoothing."
ICML,2019.用隨機噪聲填充輸入空間,得到對抗魯棒性邊界模型空間DPAbadi,Martin,etal.“Deeplearningwithdifferentialprivacy.”
CCS,
2016.差分隱私平滑模型參數(shù):DP-SGD算法DP-SGD性能SaTML2023-GautamKamath-AnIntroductiontoDifferentialPrivacyDP-SGD性能SaTML2023-GautamKamath-AnIntroductiontoDifferentialPrivacyDP-SGD性能SaTML2023-GautamKamath-AnIntroductiontoDifferentialPrivacyMore
Practical
Solution?1:
Training
on
public
dat
溫馨提示
- 1. 本站所有資源如無特殊說明,都需要本地電腦安裝OFFICE2007和PDF閱讀器。圖紙軟件為CAD,CAXA,PROE,UG,SolidWorks等.壓縮文件請下載最新的WinRAR軟件解壓。
- 2. 本站的文檔不包含任何第三方提供的附件圖紙等,如果需要附件,請聯(lián)系上傳者。文件的所有權(quán)益歸上傳用戶所有。
- 3. 本站RAR壓縮包中若帶圖紙,網(wǎng)頁內(nèi)容里面會有圖紙預(yù)覽,若沒有圖紙預(yù)覽就沒有圖紙。
- 4. 未經(jīng)權(quán)益所有人同意不得將文件中的內(nèi)容挪作商業(yè)或盈利用途。
- 5. 人人文庫網(wǎng)僅提供信息存儲空間,僅對用戶上傳內(nèi)容的表現(xiàn)方式做保護(hù)處理,對用戶上傳分享的文檔內(nèi)容本身不做任何修改或編輯,并不能對任何下載內(nèi)容負(fù)責(zé)。
- 6. 下載文件中如有侵權(quán)或不適當(dāng)內(nèi)容,請與我們聯(lián)系,我們立即糾正。
- 7. 本站不保證下載資源的準(zhǔn)確性、安全性和完整性, 同時也不承擔(dān)用戶因使用這些下載資源對自己和他人造成任何形式的傷害或損失。
最新文檔
- DB36-T1714-2022-雙低油菜“菜油兩用”栽培技術(shù)規(guī)程-江西省
- 2025年MySQL表索引題目與答案
- 門診護(hù)理禮儀培訓(xùn)
- 2025年中考數(shù)學(xué)模擬試題(數(shù)學(xué)實驗探究題)之?dāng)?shù)學(xué)實驗探究實驗論文
- 2025年學(xué)校教職工廉潔承諾書簽訂與公示要求
- 2025年一建《機電工程管理與實務(wù)》考試合同管理與索賠案例分析實戰(zhàn)演練試題
- 急診創(chuàng)傷疼痛護(hù)理
- 2025年西安市雁塔區(qū)高二年級下學(xué)期期中地理考試(自然災(zāi)害應(yīng)對與災(zāi)害管理)
- 運轉(zhuǎn)作業(yè)區(qū)2024年電工(中級工)復(fù)習(xí)試題含答案
- 財務(wù)效率提升戰(zhàn)略試題及答案
- 蘇州昆山鹿城村鎮(zhèn)銀行2023年招聘人員筆試歷年難、易錯考點試題含答案附詳解
- 山西煤炭運銷集團(tuán)錦瑞煤業(yè)有限公司煤炭資源開發(fā)利用、地質(zhì)環(huán)境保護(hù)與土地復(fù)墾方案
- 《國家中藥飲片炮制規(guī)范》全文
- 教育公共基礎(chǔ)知識整理版
- Q-SY 06351-2020 輸氣管道計量導(dǎo)則
- 鐵路工程定額電子版(Excel版)
- 如何預(yù)防與處理勞動爭議培訓(xùn)課件
- JJG 1148-2022電動汽車交流充電樁(試行)
- GB/T 31586.2-2015防護(hù)涂料體系對鋼結(jié)構(gòu)的防腐蝕保護(hù)涂層附著力/內(nèi)聚力(破壞強度)的評定和驗收準(zhǔn)則第2部分:劃格試驗和劃叉試驗
- GB/T 24917-2010眼鏡閥
評論
0/150
提交評論