🎉 Accepted to ICML 2025 🎉
Varshita Kolipaka*1, Akshit Sinha*1, Debangan Mishra1, Sumit Kumar1, Arvindh Arun1,2, Shashwat Goel†1,3,4, Ponnurangam Kumaraguru1
1IIIT Hyderabad • 2Institute for AI, University of Stuttgart • 3Max Planck Institute for Intelligent Systems • 4ELLIS Institute Tübingen
*Equal contribution. †Equal advising.
-
Install uv (if not already installed):
curl -LsSf https://astral.sh/uv/install.sh | sh -
Clone and setup environment:
git clone https://github.com/your-repo/corrective-unlearning-for-gnns cd corrective-unlearning-for-gnns uv sync
-
Create environment:
conda create --name cognac_env python=3.12 conda activate cognac_env
-
Install dependencies:
pip install -r requirements.txt
Our evaluation follows the paper's methodology with two key metrics (please refer to the paper for detailed definitions):
- Definition: Accuracy on the affected distribution (data that should be "forgotten")
- Goal: Unlearning should increase this score
- Interpretation: Higher is better - indicates better unlearning performance
- Definition: Accuracy on the remaining distribution (clean/unaffected data)
- Goal: Should remain unaffected by unlearning
- Interpretation: Higher is better - indicates preserved model utility
- Forget: Higher = Better unlearning
- Utility: Higher = Better retention of original performance
- The ideal unlearning method maximizes both metrics
Use this script to automatically run both Cognac variants and get the best result:
python run_cognac_best.py --dataset Cora --gnn gcn --df_size 0.3 --attack_type labelThis script will:
- Run hyperparameter tuning for both
cognacandcognac-descent - Run main experiments for both variants
- Compare results and report the best performing variant
- Provide a clear recommendation for your specific configuration
You must run this first to find optimal hyperparameters:
python hp_tune.py --dataset Cora --gnn gcn --attack_type label --unlearning_model cognacAfter hyperparameter tuning, run the main experiments:
python main.py --dataset Cora --gnn gcn --df_size 0.3 --attack_type label --unlearning_model cognacWe propose Cognac, which comes in two variants:
cognac: Full method with ascent and descentcognac-descent: Descent-only variant
The run_cognac_best.py script automatically tests both and recommends the best one for your configuration.
Manual commands for individual variants:
# Run Cognac (full method)
python main.py --dataset Cora --gnn gcn --df_size 0.3 --attack_type label --unlearning_model cognac
# Run Cognac-Descent (descent only)
python main.py --dataset Cora --gnn gcn --df_size 0.3 --attack_type label --unlearning_model cognac-descentcognac: Full Cognac method with ascent and descent phasescognac-descent: Cognac descent-only variant
retrain: Complete retraining from scratchgnndelete: GNNDelete unlearning methodgradient_ascent: Simple gradient ascent approachscrub: SCRUB unlearning methodmegu: MEGU unlearning approachssd: Selective Synaptic Dampeninggif: GIFutu: Unlink to Unlearnacdc: Ascent-Descent
python main.py [OPTIONS]--dataset: Dataset to use (e.g.,Cora,CiteSeer,PubMed)--gnn: GNN architecture (gcn,gat,gin)--attack_type: Attack method (label,edge,trigger)--unlearning_model: Unlearning method (see available methods above)--df_size: Forgetting fraction (e.g.,0.3for 30%)
# Recommended: Automated best variant selection
python run_cognac_best.py --dataset Cora --gnn gcn --df_size 0.3 --attack_type label
# Manual: Basic Cognac run
python main.py --dataset Cora --gnn gcn --df_size 0.3 --attack_type label --unlearning_model cognac
# Manual: Cognac-descent variant
python main.py --dataset Cora --gnn gcn --df_size 0.3 --attack_type label --unlearning_model cognac-descent
# Baseline comparison
python main.py --dataset Cora --gnn gcn --df_size 0.3 --attack_type label --unlearning_model retrainFor complete argument list, run: python main.py --help
The run_cognac_best.py script automatically finds the best Cognac variant for your configuration:
# Basic usage - minimal arguments
python run_cognac_best.py --dataset Cora --gnn gcn --attack_type label --df_size 0.3
# Full configuration for research paper reproduction
python run_cognac_best.py \
--dataset Cora \
--gnn gcn \
--attack_type label \
--df_size 0.3 \
--random_seed 42 \
--train_ratio 0.6 \
--val_ratio 0.2 \
--hidden_dim 64 \
--training_epochs 1208 \
--unlearning_epochs 200
# Different datasets and attack types
python run_cognac_best.py --dataset CiteSeer --gnn gat --attack_type edge --df_size 0.5
python run_cognac_best.py --dataset PubMed --gnn gin --attack_type trigger --df_size 0.2
# Advanced control options
python run_cognac_best.py --dataset Cora --gnn gcn --attack_type label --df_size 0.3 --skip-hp-tune # Skip HP tuning
python run_cognac_best.py --dataset Cora --gnn gcn --attack_type label --df_size 0.3 --only-compare # Only compare existing resultsFor researchers who want full control over each step:
# Tune Cognac (full method)
python hp_tune.py \
--dataset Cora \
--gnn gcn \
--attack_type label \
--unlearning_model cognac \
--df_size 0.3 \
--random_seed 42
# Tune Cognac-Descent (descent only)
python hp_tune.py \
--dataset Cora \
--gnn gcn \
--attack_type label \
--unlearning_model cognac-descent \
--df_size 0.3 \
--random_seed 42# Run Cognac (full method)
python main.py \
--dataset Cora \
--gnn gcn \
--attack_type label \
--unlearning_model cognac \
--df_size 0.3 \
--random_seed 42 \
--train_ratio 0.6 \
--val_ratio 0.2 \
--hidden_dim 64 \
--training_epochs 1208 \
--unlearning_epochs 200
# Run Cognac-Descent (descent only)
python main.py \
--dataset Cora \
--gnn gcn \
--attack_type label \
--unlearning_model cognac-descent \
--df_size 0.3 \
--random_seed 42 \
--train_ratio 0.6 \
--val_ratio 0.2 \
--hidden_dim 64 \
--training_epochs 1208 \
--unlearning_epochs 200# Retrain baseline
python hp_tune.py --dataset Cora --gnn gcn --attack_type label --unlearning_model retrain --df_size 0.3
python main.py --dataset Cora --gnn gcn --attack_type label --unlearning_model retrain --df_size 0.3
# GNNDelete baseline
python hp_tune.py --dataset Cora --gnn gcn --attack_type label --unlearning_model gnndelete --df_size 0.3
python main.py --dataset Cora --gnn gcn --attack_type label --unlearning_model gnndelete --df_size 0.3
# SCRUB baseline
python hp_tune.py --dataset Cora --gnn gcn --attack_type label --unlearning_model scrub --df_size 0.3
python main.py --dataset Cora --gnn gcn --attack_type label --unlearning_model scrub --df_size 0.3
# MEGU baseline
python hp_tune.py --dataset Cora --gnn gcn --attack_type label --unlearning_model megu --df_size 0.3
python main.py --dataset Cora --gnn gcn --attack_type label --unlearning_model megu --df_size 0.3For comprehensive evaluation across multiple settings:
# Multiple datasets
for dataset in Cora CiteSeer PubMed; do
python run_cognac_best.py --dataset $dataset --gnn gcn --attack_type label --df_size 0.3
done
# Multiple attack types
for attack in label edge trigger; do
python run_cognac_best.py --dataset Cora --gnn gcn --attack_type $attack --df_size 0.3
done
# Multiple forgetting fractions
for df_size in 0.1 0.3 0.5; do
python run_cognac_best.py --dataset Cora --gnn gcn --attack_type label --df_size $df_size
done
# Multiple GNN architectures
for gnn in gcn gat gin; do
python run_cognac_best.py --dataset Cora --gnn $gnn --attack_type label --df_size 0.3
done
# Multiple random seeds for statistical significance
for seed in 42 123 456 789 999; do
python run_cognac_best.py --dataset Cora --gnn gcn --attack_type label --df_size 0.3 --random_seed $seed
doneResults will be saved in the following structure:
logs/default/
├── Cora/
│ ├── run_logs_label_0.3_42.json # Main results
│ ├── run_logs_label_0.3_123.json # Different seed
│ └── ...
├── CiteSeer/
│ └── ...
└── PubMed/
└── ...
hp_tuning/
└── hp_tuning.db # Hyperparameter optimization results
best_params.json # Best hyperparameters for each configuration
When baselining, focus on these metrics from the log files:
- Test Accuracy: Overall model performance
- Forget Accuracy: Performance on nodes that should be forgotten
- Utility Accuracy: Performance on remaining nodes
- F1 Scores: For imbalanced datasets
- Training Time: Computational efficiency
attacks/: Graph attack implementationsframework/: Core utilities and training argumentsmodels/: GNN architectures (GCN, GAT, GIN)trainers/: Unlearning method implementationshp_tune.py: Required hyperparameter optimizationmain.py: Main experiment runnerrun_cognac_best.py: Recommended automated script to find best Cognac variantpyproject.toml: uv project configurationrequirements.txt: Alternative dependency list
@misc{kolipaka2024cognacshotforgetbad,
title={A Cognac shot to forget bad memories: Corrective Unlearning in GNNs},
author={Varshita Kolipaka and Akshit Sinha and Debangan Mishra and Sumit Kumar and Arvindh Arun and Shashwat Goel and Ponnurangam Kumaraguru},
year={2024},
eprint={2412.00789},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2412.00789},
}- Hyperparameter tuning is mandatory before running experiments
- Use
run_cognac_best.pyfor automatic best variant selection (recommended) - For manual runs, test both
cognacandcognac-descentvariants for fair comparison - Results are stored in the
logs/directory - The project uses uv for modern Python dependency management
- GPU is recommended for faster training