-
ProjB: An Improved Bilinear Biased ProjE model for Knowledge Graph Completion
Authors:
Mojtaba Moattari,
Sahar Vahdati,
Farhana Zulkernine
Abstract:
Knowledge Graph Embedding (KGE) methods have gained enormous attention from a wide range of AI communities including Natural Language Processing (NLP) for text generation, classification and context induction. Embedding a huge number of inter-relationships in terms of a small number of dimensions, require proper modeling in both cognitive and computational aspects. Recently, numerous objective fun…
▽ More
Knowledge Graph Embedding (KGE) methods have gained enormous attention from a wide range of AI communities including Natural Language Processing (NLP) for text generation, classification and context induction. Embedding a huge number of inter-relationships in terms of a small number of dimensions, require proper modeling in both cognitive and computational aspects. Recently, numerous objective functions regarding cognitive and computational aspects of natural languages are developed. Among which are the state-of-the-art methods of linearity, bilinearity, manifold-preserving kernels, projection-subspace, and analogical inference. However, the major challenge of such models lies in their loss functions that associate the dimension of relation embeddings to corresponding entity dimension. This leads to inaccurate prediction of corresponding relations among entities when counterparts are estimated wrongly. ProjE KGE, published by Bordes et al., due to low computational complexity and high potential for model improvement, is improved in this work regarding all translative and bilinear interactions while capturing entity nonlinearity. Experimental results on benchmark Knowledge Graphs (KGs) such as FB15K and WN18 show that the proposed approach outperforms the state-of-the-art models in entity prediction task using linear and bilinear methods and other recent powerful ones. In addition, a parallel processing structure is proposed for the model in order to improve the scalability on large KGs. The effects of different adaptive clustering and newly proposed sampling approaches are also explained which prove to be effective in improving the accuracy of knowledge graph completion.
△ Less
Submitted 15 September, 2022; v1 submitted 15 August, 2022;
originally announced September 2022.
-
Uncertainty Principle based optimization; new metaheuristics framework
Authors:
Mojtaba Moattari,
Mohammad Hassan Moradi,
Emad Roshandel
Abstract:
To more flexibly balance between exploration and exploitation, a new meta-heuristic method based on Uncertainty Principle concepts is proposed in this paper. UP is is proved effective in multiple branches of science. In the branch of quantum mechanics, canonically conjugate observables such as position and momentum cannot both be distinctly determined in any quantum state. In the same manner, the…
▽ More
To more flexibly balance between exploration and exploitation, a new meta-heuristic method based on Uncertainty Principle concepts is proposed in this paper. UP is is proved effective in multiple branches of science. In the branch of quantum mechanics, canonically conjugate observables such as position and momentum cannot both be distinctly determined in any quantum state. In the same manner, the branch of Spectral filtering design implies that a nonzero function and its Fourier transform cannot both be sharply localized. After delving into such concepts on Uncertainty Principle and their variations in quantum physics, Fourier analysis, and wavelet design, the proposed framework is described in terms of algorithm and flowchart. Our proposed optimizer's idea is based on an inherent uncertainty in performing local search versus global solution search. A set of compatible metrics for each part of the framework is proposed to derive preferred form of algorithm. Evaluations and comparisons at the end of paper show competency and distinct capability of the algorithm over some of the well-known and recently proposed metaheuristics.
△ Less
Submitted 2 June, 2020;
originally announced June 2020.
-
Modified swarm-based metaheuristics enhance Gradient Descent initialization performance: Application for EEG spatial filtering
Authors:
Mojtaba Moattari,
Mohammad Hassan Moradi,
Reza Boostani
Abstract:
Gradient Descent (GD) approximators often fail in the solution space with multiple scales of convexities, i.e., in subspace learning and neural network scenarios. To handle that, one solution is to run GD multiple times from different randomized initial states and select the best solution over all experiments. However, this idea is proved impractical in plenty of cases. Even Swarm-based optimizers…
▽ More
Gradient Descent (GD) approximators often fail in the solution space with multiple scales of convexities, i.e., in subspace learning and neural network scenarios. To handle that, one solution is to run GD multiple times from different randomized initial states and select the best solution over all experiments. However, this idea is proved impractical in plenty of cases. Even Swarm-based optimizers like Particle Swarm Optimization (PSO) or Imperialistic Competitive Algorithm (ICA), as commonly used GD initializers, have failed to find optimal solutions in some applications. In this paper, Swarm-based optimizers like ICA and PSO are modified by a new optimization framework to improve GD optimization performance. This improvement is for applications with high number of convex localities in multiple scales. Performance of the proposed method is analyzed in a nonlinear subspace filtering objective function over EEG data. The proposed metaheuristic outperforms commonly used baseline optimizers as GD initializers in both the EEG classification accuracy and EEG loss function fitness. The optimizers have been also compared to each other in some of CEC 2014 benchmark functions, where again our method outperforms other algorithms.
△ Less
Submitted 6 May, 2020; v1 submitted 13 June, 2019;
originally announced July 2019.
-
A New Approach for Optimizing Highly Nonlinear Problems Based on the Observer Effect Concept
Authors:
Mojtaba Moattari,
Emad Roshandel,
Shima Kamyab,
Zohreh Azimifar
Abstract:
A lot of real-world engineering problems represent dynamicity with nests of nonlinearities due to highly complex network of exponential functions or large number of differential equations interacting together. Such search spaces are provided with multiple convex regions peaked with diverse nonlinear slopes and in non-homogenous ways. To find global optima, a new meta-heuristic algorithm is propose…
▽ More
A lot of real-world engineering problems represent dynamicity with nests of nonlinearities due to highly complex network of exponential functions or large number of differential equations interacting together. Such search spaces are provided with multiple convex regions peaked with diverse nonlinear slopes and in non-homogenous ways. To find global optima, a new meta-heuristic algorithm is proposed based on Observer Effect concepts for controlling memory usage per localities without pursuing Tabu-like cut-off approaches. Observer effect in physics (or psychology) regards bias in measurement (or perception) due to the interference of instrument (or knowledge). Performance analysis of the proposed algorithms is sought in two real-world engineering applications, i.e., Electroencephalogram feature learning and Distributed Generator parameter tuning, each of which having nonlinearity and complex multi-modal peaks distributions as their characteristics. In addition, the effect of version improvement has been assessed. The performance comparison with other optimizers in the same context suggests that proposed algorithm is useful both solely and in hybrid Gradient Descent settings where problem's search space is nonhomogeneous in terms of local peaks density.
△ Less
Submitted 6 May, 2020; v1 submitted 13 June, 2019;
originally announced June 2019.
