This repository contains a PyTorch implementation of "FDGATII : Fast Dynamic Graph Attention with Initial Residual and Identity Mapping".(https://arxiv.org/abs/2110.11464)

FDGATII combines 3 main enhancements on GAT, namely a default (safety net) initial representation is provided that is used when no similar class or node is present in the neighbourhood, so that graph attention mechanism will not fail. A more expressive universal dynamic attention is used. Finally, string regularization is applied, controlled by the hyperparameter β.

The repo, inlcuding data and datasplits used for the 10 ierations, has been forked initially from GCNII. We use the sparse (static) GATv1 attention code from pyGAT and modified dynamic attention as in GATv2.

• CUDA 11.3.0
• python 3.6.9
• pytorch 1.3.1

Note : FDGATII is able to run with no GPU if the GUP timing code is commented out, and then will not require CUDA.

The data folder contains three benchmark datasets(Cora, Citeseer, Pubmed), and the new_data folder contains four datasets(Chameleon, Cornell, Texas, Wisconsin) from Geom-GCN. We use the same same full-supervised setting, and data splits, as Geom-GCN and GCNII.

Testing accuracy is summarized below. We have used the 10 standard data splits ( Split 0 - 9) and obtained the Avarage Accuracy and Standard deviation.

• All parameters are defined in fullSupervised_01.py.

• To run FDGATII on cora, for 1 iteration only use

python -u fullSupervised_01.py --data cora --layer 2 --alpha 0.2 --weight_decay 1e-4 --epochs 1500 --iterations 1 --mode FDGATII --support 1 --verbosity 1 --model GCNII_BASE

• To replicate the FDGATII full-supervised results, run the following script

#!/bin/bash
SCRIPT='python3.6 fullSupervised_01.py'
SETTTINGS=" --epochs 1500 --iterations 10 --mode FDGATII --verbosity 0 --model GCNII_BASE "

$SCRIPT --data cora --support 1 --layer 2 --hidden 64 --alpha 0.2 --weight_decay 1e-4 $SETTTINGS
$SCRIPT --data citeseer --support 2 --layer 1 --hidden 128  --weight_decay 5e-6 $SETTTINGS
$SCRIPT --data pubmed --support 1 --layer 2 --hidden 64  --alpha 0.1 --weight_decay 5e-6 $SETTTINGS
$SCRIPT --data chameleon --support 0 --layer 1 --hidden 64 --lamda 1.5 --alpha 0.2 --weight_decay 5e-4 $SETTTINGS
$SCRIPT --data cornell --support 2 --layer 1 --hidden  128 --lamda 1 --weight_decay 1e-3 $SETTTINGS
$SCRIPT --data texas --support 2 --layer 2 --hidden 64 --lamda 1.5 --weight_decay 1e-4 $SETTTINGS
$SCRIPT --data wisconsin --support 2 --layer 1 --hidden 128 --lamda 1 --weight_decay 5e-4 $SETTTINGS

Datasets and code is forked from GCNII which uses the data set and parts of code from Geom-GCN. We use sparse attention implementation code from pyGAT as discribed in GAT . We acknowledge and thank the authors of these works for sharing their code.

@article{kulatilleke2021fdgatii,
  title={FDGATII: Fast Dynamic Graph Attention with Initial Residual and Identity Mapping},
  author={Kulatilleke, Gayan K and Portmann, Marius and Ko, Ryan and Chandra, Shekhar S},
  journal={arXiv preprint arXiv:2110.11464},
  year={2021}
}