6.4 Chapter Summary
Key Takeaways
- Privacy Attacks exploit vulnerabilities in datasets/models to infer sensitive data or membership.
- Federated Learning (FL) decentralizes training, keeping data on user devices and sharing only model parameters.
- Trust models define the level of trust among data owners, system providers, consumers, and potential adversaries. Attacks often occur when these roles are distributed.
- Data Reconstruction Attacks:
- Reverse aggregated data to recover individual records (e.g., Dinur-Nissim attacks).
- Model inversion and memorization in deep learning worsen risks.
- Membership Inference Attacks:
- Determine if a record was in the training set (e.g., healthcare data leaks).
- Techniques: Loss-based, shadow models, LiRA, label-only attacks.
- Model Extraction Attacks:
- Clone proprietary models via queries (e.g., MLaaS).
- Methods: Mathematical extraction, learning-based, side-channel attacks.
- Mitigation:
- Differential Privacy (DP): Noise injection (Gaussian/Laplace mechanisms).
- DP-SGD: Privacy-preserving neural network training.
- PPML: Homomorphic encryption, secure multi-party computation.
- Challenges: Balancing privacy-utility trade-offs; there is no one-size-fits-all solution to privacy attacks; robust defence requires a combination of strategies tailored to specific threat models.
Key Terms
- Data reconstruction attacks are the most concerning privacy attacks as they have the ability to recover an individual’s data from released aggregate information, focusing on reversing aggregated statistical data to recover individual records.
- Differentiated privacy (DP) introduces controlled noise into query responses or training processes, limiting how individual records can influence model outputs.
- Inference-Based Attacks exploit patterns in data distributions and model outputs to extract sensitive information.
- Membership inference attacks aim to determine whether a specific record was part of a training dataset.
- Model extraction attacks seek to replicate proprietary machine-learning models by analyzing their responses to input queries.
- Property inference attacks seek to deduce aggregate dataset attributes, such as demographic distributions or class imbalances
