Embedding Knowledge Graph through Triple Base Neural Network and Positive Samples

Document Type : Semantic Technology-Kahani

Authors

Sogol Haghani

Mohammad Reza Keyvanpour

Department of Computer Engineering, Alzahra University, Tehran, Iran.

10.22067/cke.2022.73802.1038

Abstract

Representation learning on a knowledge graph aims to capture patterns in the knowledge graph as low-dimensional dense distributed representation vectors in the continuous semantic space, which is a powerful technique for predicting missing links in knowledge bases. The problem of knowledge base completion can be viewed as predicting new triples based on the existing ones. One of the prominent approaches in knowledge base completion is the embedding model. Currently, the majority of existing knowledge graph embedding models cannot deal with unbalanced entities and relations. In this paper, a new embedding model is proposed, with a general solution instead of using the additional corpus. First, a triple-based neural network is presented to maximize the likelihood of the knowledge bases finding a low-dimensional embedding space. Second, two procedures to generate positive triples are proposed. They produce positive triples and add them to the training data. The policies can capture rare triples, and simultaneously remain efficient to compute. Experiments show that the embedded model proposed in this paper has superior performance.

Keywords

Knowledge Graphs

Link Prediction

Positive Samples

Embedding Neural Network

Graph mining

Subjects

• Semantic Technology

---

References

[1]   Miller, G. A., "WordNet: a lexical database for English," Communications of the ACM, pp. 39-41, 1995.

[2]   Suchanek, F. M., Kasneci, G., and Weikum, G., "Yago: a core of semantic knowledge," 2007.

[3]   Nickel, M., Murphy, K., Tresp, V., and Gabrilovich, E., "A review of relational machine learning for knowledge graphs," Proceedings of the IEEE, Vol. 104, No. 1, pp. 11-33, 2015.

[4]   Sadeghi, A., Graux, D., and Lehmann, J., "MDE: Multi Distance Embeddings for Link Prediction in Knowledge Graphs," arXiv preprint arXiv:1905.10702, 2019.

[5]   Cai, H., Zheng, V. W., and Chang, K., "A comprehensive survey of graph embedding: Problems, techniques and applications," IEEE Transactions on Knowledge and Data Engineering, p. IEEE, 2018.

[6]   Ebisu, T., and Ichise, R., "Toruse: Knowledge graph embedding on a lie group," in Thirty-Second AAAI Conference on Artificial Intelligence, 2018.

[7]   Mighan, Sima Naderi, Mohsen Kahani, and Fateme Pourgholamali. "POI Recommendation Based on Heterogeneous Graph Embedding." 2019 9th International Conference on Computer and Knowledge Engineering (ICCKE). IEEE, 2019.‏

[8]   Nguyen, D. Q., Sirts, K., Qu, L., and Johnson, M., "STransE: a novel embedding model of entities and relationships in knowledge bases," in Proceedings of NAACL-HLT, 2016.

[9]   Ji, G., Liu, K., He, S., and Zhao, J., "Knowledge graph completion with adaptive sparse transfer matrix," in Thirtieth AAAI Conference on Artificial Intelligence, 2016.

[10] Wang, Z., Zhang, J., Feng, J., and Chen, Z., "Knowledge Graph Embedding by Translating on Hyperplanes," in AAAI, 2014.

[11] Li, Zhifei, Hai Liu, Zhaoli Zhang, Tingting Liu, and Neal N. Xiong. "Learning knowledge graph embedding with heterogeneous relation attention networks." IEEE Transactions on Neural Networks and Learning Systems, 2021.

[12] Trouillon, T., Welbl, J., Riedel, S., Gaussier, E., and Bouchard, G., "Complex embeddings for simple link prediction," in International Conference on Machine Learning, 2016.

[13] Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., and Yakhnenko, O., "Translating embeddings for modeling multi-relational data," in Advances in neural information processing systems, 2013.

[14] Lin, Y., Liu, Z., Sun, M., Liu, Y., and Zhu, X., "Learning Entity and Relation Embeddings for Knowledge Graph Completion," in AAAI, 2015.

[15] Yang, B., Yih, W.-t., He, X., Gao, J., and Deng, L., "Embedding entities and relations for learning and inference in knowledge bases," arXiv preprint arXiv:1412.6575, 2014.

[16] Socher, R., Chen, D., Manning, C. D., and Ng, A., "Reasoning with neural tensor networks for knowledge base completion," in Advances in neural information processing systems, 2013.

[17] Abedini, F., Menhaj, M. B., and Keyvanpour, M. R., "Neuron Mathematical Model Representation of Neural Tensor Network for RDF Knowledge Base Completion," Journal of Computer & Robotics, Vol. 10, No. 1, pp. 1-10, 2017.

[18] Nickel, M., Rosasco, L., Poggio, T. A., and others, "Holographic Embeddings of Knowledge Graphs," AAAI, pp. 1955-1961, 2016.

[19] Haghani, S., and Keyvanpour, M. R., "moLink: Modeling link representation of knowledge base," in Information and Knowledge Technology (IKT), 2017 9th International Conference on, 2018.

[20] Cao, X., Wei, Y., Wen, F., and Sun, J., "Face alignment by explicit shape regression," International Journal of Computer Vision, Vol. 107, No. 2, pp. 177-190, 2014.

[21] Cui, X., Goel, V., and Kingsbury, B., "Data augmentation for deep neural network acoustic modeling," IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), Vol. 23, No. 9, pp. 1469-1477, 2015.

[22] Bengio, Y., Courville, A., and Vincent, P., "Representation learning: A review and new perspectives," IEEE transactions on pattern analysis and machine intelligence, Vol. 35, No. 8, pp. 1798-1828, 2013.

[23] Haghani, S., and Keyvanpour, M. R., "A systemic analysis of link prediction in social network," Artificial Intelligence Review, Vol. 52, pp. 1961-1995, 2019.

[24] Keyvanpour, M., Kholghi, M., and Haghani, S., "Hybrid of Active Learning and Dynamic Self-Training for Data Stream Classification," International Journal of Information & Communication Technology Research, Vol. 9, No. 4, 2017.

[25] Zhu, J., Jia, Y., Xu, J., and others, "Modeling the Correlations of Relations for Knowledge Graph Embedding," J. Comput. Sci. & Technol, Vol. 33, No. 2, 2018.

[26] Chang, L., Zhu, M., Gu, T., Bin, C., Qian, J., and Zhang, J., "Knowledge Graph Embedding by Dynamic Translation," IEEE Access, Vol. 5, pp. 20898-20907, 2017.

[27] Xiao, H., Huang, M., and Zhu, X., "TransG: A generative model for knowledge graph embedding," in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016.

[28] Glorot, X., and Bengio, Y., "Understanding the difficulty of training deep feedforward neural networks," in Proceedings of the thirteenth international conference on artificial intelligence and statistics, 2010.

[29] Xiao, H., Huang, M., and Zhu, X., "From one point to a manifold: Knowledge graph embedding for precise link prediction," arXiv preprint arXiv:1512.04792, 2015.

[30] Rosso, Paolo, Dingqi Yang, and Philippe Cudré-Mauroux. "Beyond triplets: hyper-relational knowledge graph embedding for link prediction." Proceedings of The Web Conference 2020.

[31] Dettmers, T., Minervini, P., Stenetorp, P., and Riedel, S., "Convolutional 2d knowledge graph embeddings," in Thirty-Second AAAI Conference on Artificial Intelligence, 2018.

[32] Schlichtkrull, M. a. K. T. N., Bloem, P., van den Berg, R., Titov, I., and Welling, M., "Modeling relational data with graph convolutional networks," in European Semantic Web Conference, 2018.

[33] Yu, Donghan, et al. "Knowledge embedding based graph  convolutional network." Proceedings of the Web Conference 2021. 2021

[34] Kazemi, Seyed Mehran, and David Poole. "Simple embedding for link prediction in knowledge graphs." Advances in neural information processing systems 31, 2018.

[35] Sun, Zhiqing, et al. "Rotate: Knowledge graph embedding by relational rotation in complex space." arXiv preprint arXiv:1902.10197, 2019.

[36] Nguyen, Dat Quoc. "An overview of embedding models of entities and relationships for knowledge base completion." arXiv preprint arXiv:1703.08098, 2017