Kashob Kumar Roy
- Research Assistant, CCDS
PhD Student
University of Illinois Urbana-Champaign
Research Interests
Machine Learning
Conference Paper2021Cited by 1
Node embedding using mutual information and self-supervision based bi-level aggregation
Kashob Kumar Roy, Amit Roy, AKM Mahbubur Rahman, M Ashraful Amin, Amin Ahsan Ali
2021 International Joint Conference on Neural Networks (IJCNN)
IEEE, pp. 1–8
Abstract
Graph Neural Networks (GNNs) learn low dimensional representations of nodes by aggregating information from their neighborhood in graphs. However, traditional GNNs suffer from two fundamental shortcomings due to their local (l-hop neighborhood) aggregation scheme. First, not all nodes in the neighborhood carry relevant information for the target node. Since GNNs do not exclude noisy nodes in their neighborhood, irrelevant information gets aggregated, which reduces the quality of the representation. Second, traditional GNNs also fail to capture long-range non-local dependencies between nodes. To address these limitations, we exploit mutual information (MI) to define two types of neighborhood, 1) Local Neighborhood where nodes are densely connected within a community and each node would share higher MI with its neighbors, and 2) Non-Local Neighborhood where MI-based node clustering is introduced to assemble informative but graphically distant nodes in the same cluster. To generate node presentations, we combine the embeddings generated by bi-level aggregation - local aggregation to aggregate features from local neighborhoods to avoid noisy information and non-local aggregation to aggregate features from non-local neighborhoods. Furthermore, we leverage self-supervision learning to estimate MI with few labeled data. Finally, we show that our model significantly outperforms the state-of-the-art methods in a wide range of assortative and disassortative graphs11Source Code at: https://github.com/forkkr/LnL-GNN.
Keywords
Computer scienceAggregate (composite)ExploitLeverage (statistics)Node (physics)EmbeddingCluster analysisTheoretical computer scienceGraphRange (aeronautics)Data mining
CCDS Authors
Amit Roy
- Research Assistant, CCDS
PhD Student
Purdue University
Research Interests
Machine Learning
AKM Mahbubur Rahman, PhD
- Director, DS & IPT Wings
Associate Professor
Department of Computer Science and Engineering
Independent University, Bangladesh
Prof. M Ashraful Amin, PhD
- Founder Director, CCDS
- Director, AI/ML, HCI & DS Wings
Professor
+1 more affiliationDepartment of Information Sciences and Technology
George Mason University, USA
Research Interests
Machine Learning, Cognitive & Vision Science, Cybernetics, Surveillance & Security, ICT in Education, Health, & Agriculture, Human-Computer Interaction, Internet of Things, Robotics
Amin Ahsan Ali, PhD
- Director, AI/ML Wing
Professor
Department of Computer Science and Engineering
Independent University, Bangladesh
Research Interests
Artificial Intelligence, Machine Learning
References
- 1.Vincent D. Blondel, Jean‐Loup Guillaume, Renaud Lambiotte, Etienne Lefebvre. (2008). Fast unfolding of communities in large networks. Journal of Statistical Mechanics Theory and Experiment, 2008(10), P10008[10.1088/1742-5468/2008/10/p10008]
- 2.Chen, Tianyu, Wang, Zeyu, Li, Lin, Li, Ding, Li, Zongyang, Chang, Xiaoning, Bian, Pan, Liang, Guangtai, Wang, Qianxiang, Xie, Tao. (2025). Detecting Functionality-Specific Vulnerabilities via Retrieving Individual Functionality-Equivalent APIs in Open-Source Repositories. Dagstuhl Research Online Publication Server[10.4230/lipics.ecoop.2025.6]
- 3.M. E. J. Newman. (2006). Modularity and community structure in networks. Proceedings of the National Academy of Sciences, 103(23), 8577–8582[10.1073/pnas.0601602103]
- 4.Qimai Li, Zhichao Han, Xiao-Ming Wu. (2018). Deeper Insights Into Graph Convolutional Networks for Semi-Supervised Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1)[10.1609/aaai.v32i1.11604]
- 5.M. E. J. Newman. (2004). Analysis of weighted networks. Physical Review E, 70(5), 056131[10.1103/physreve.70.056131]
- 6.Guohao Li, Matthias Müller, Ali Thabet, Bernard Ghanem. (2019). DeepGCNs: Can GCNs Go As Deep As CNNs?. , 9266–9275[10.1109/iccv.2019.00936]
- 7.Deli Chen, Yankai Lin, Wei Li, Peng Li, Jie Zhou, Xu Sun. (2020). Measuring and Relieving the Over-Smoothing Problem for Graph Neural Networks from the Topological View. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04), 3438–3445[10.1609/aaai.v34i04.5747]
- 8.Xiao Wang, Peng Cui, Jing Wang, Jian Pei, Wenwu Zhu, Shiqiang Yang. (2017). Community Preserving Network Embedding. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1)[10.1609/aaai.v31i1.10488]
- 9.Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, Devon Hjelm. (2018). Mutual Information Neural Estimation.. International Conference on Machine Learning, 531–540
- 10.Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken‐ichi Kawarabayashi, Stefanie Jegelka. (2018). Representation Learning on Graphs with Jumping Knowledge Networks. International Conference on Machine Learning, 5453–5462
- 11.Di Jin, Zhizhi Yu, Pengfei Jiao, Shirui Pan, Dongxiao He, Jia Wu, Philip L. H. Yu, Weixiong Zhang. (2021). A Survey of Community Detection Approaches: From Statistical Modeling to Deep Learning. IEEE Transactions on Knowledge and Data Engineering, 1[10.1109/tkde.2021.3104155]
- 12.Yuning You, Tianlong Chen, Zhangyang Wang, Yang Shen. (2020). When Does Self-Supervision Help Graph Convolutional Networks?. PubMed, 119, 10871–10880[link]
- 13.Meng Liu, Zhengyang Wang, Shuiwang Ji. (2021). Non-Local Graph Neural Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12), 10270–10276[10.1109/tpami.2021.3134200]
- 14.Pengyang Wang, Yanjie Fu, Yuanchun Zhou, Kunpeng Liu, Xiaolin Li, Kien A. Hua. (2020). Exploiting Mutual Information for Substructure-aware Graph Representation Learning. , 3415–3421[10.24963/ijcai.2020/472]
- 15.R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, Yoshua Bengio. (2018). Learning deep representations by mutual information estimation and maximization. arXiv (Cornell University)[10.48550/arxiv.1808.06670]
- 16.Yu Rong, Wenbing Huang, Tingyang Xu, Junzhou Huang. (2019). DropEdge: Towards Deep Graph Convolutional Networks on Node Classification. arXiv (Cornell University)[10.48550/arxiv.1907.10903]
- 17.Petar Veličković, William Fedus, William L. Hamilton, Píetro Lió, Yoshua Bengio, R Devon Hjelm. (2018). Deep Graph Infomax. Apollo (University of Cambridge)[10.17863/cam.40744]
- 18.Hongbin Pei, Bingzhe Wei, Kevin Chen–Chuan Chang, Lei Yu, Bo Yang. (2020). Geom-GCN: Geometric Graph Convolutional Networks. arXiv (Cornell University)[10.48550/arxiv.2002.05287]
- 19.Di Jin, Zhizhi Yu, Pengfei Jiao, Shirui Pan, Dongxiao He, Jia Wu, Philip S. Yu, Weixiong Zhang. (2021). A Survey of Community Detection Approaches: From Statistical Modeling to Deep Learning. arXiv (Cornell University)[10.48550/arxiv.2101.01669]
Cited By
- 1.Sun, Feng, Kumar, Ajith, Yang, Guanci, Zhu, Qikui, Zhang, Yiyun, Zhang, Ansi, Makwana, Dhruv. (2021). RRLFSOR: An Efficient Self-Supervised Learning Strategy of Graph Convolutional Networks. arXiv (Cornell University)[10.48550/arxiv.2108.07481]