Showing 1–20 of 20 results for author: Saad, D

Scalable Node-Disjoint and Edge-Disjoint Multi-wavelength Routing

Authors: Yi-Zhi Xu, Ho Fai Po, Chi Ho Yeung, David Saad

Abstract: Probabilistic message-passing algorithms are developed for routing transmissions in multi-wavelength optical communication networks, under node and edge-disjoint routing constraints and for various objective functions. Global routing optimization is a hard computational task on its own but is made much more difficult under the node/edge-disjoint constraints and in the presence of multiple waveleng… ▽ More Probabilistic message-passing algorithms are developed for routing transmissions in multi-wavelength optical communication networks, under node and edge-disjoint routing constraints and for various objective functions. Global routing optimization is a hard computational task on its own but is made much more difficult under the node/edge-disjoint constraints and in the presence of multiple wavelengths, a problem which dominates routing efficiency in real optical communication networks that carry most of the world's Internet traffic. The scalable principled method we have developed is exact on trees but provides good approximate solutions on locally tree-like graphs. It accommodates a variety of objective functions that correspond to low latency, load balancing and consolidation of routes, and can be easily extended to include heterogeneous signal-to-noise values on edges and a restriction on the available wavelengths per edge. It can be used for routing and managing transmissions on existing topologies as well as for designing and modifying optical communication networks. Additionally, it provides the tool for settling an open and much debated question on the merit of wavelength-switching nodes and the added capabilities they provide. The methods have been tested on generated networks such as random-regular, Erdős Rényi and power-law graphs, as well as on the UK and US optical communication networks. They show excellent performance with respect to existing methodology on small networks and have been scaled up to network sizes that are beyond the reach of most existing algorithms. △ Less

Submitted 1 July, 2021; originally announced July 2021.

COLLIDE-PRED: Prediction of On-Road Collision From Surveillance Videos

Authors: Deesha Chavan, Dev Saad, Debarati B. Chakraborty

Abstract: Predicting on-road abnormalities such as road accidents or traffic violations is a challenging task in traffic surveillance. If such predictions can be done in advance, many damages can be controlled. Here in our wok, we tried to formulate a solution for automated collision prediction in traffic surveillance videos with computer vision and deep networks. It involves object detection, tracking, tra… ▽ More Predicting on-road abnormalities such as road accidents or traffic violations is a challenging task in traffic surveillance. If such predictions can be done in advance, many damages can be controlled. Here in our wok, we tried to formulate a solution for automated collision prediction in traffic surveillance videos with computer vision and deep networks. It involves object detection, tracking, trajectory estimation, and collision prediction. We propose an end-to-end collision prediction system, named as COLLIDE-PRED, that intelligently integrates the information of past and future trajectories of moving objects to predict collisions in videos. It is a pipeline that starts with object detection, which is used for object tracking, and then trajectory prediction is performed which concludes by collision detection. The probable place of collision, and the objects those may cause the collision, both can be identified correctly with COLLIDE-PRED. The proposed method is experimentally validated with a number of different videos and proves to be effective in identifying accident in advance. △ Less

Submitted 21 January, 2021; originally announced January 2021.

Space of Functions Computed by Deep-Layered Machines

Authors: Alexander Mozeika, Bo Li, David Saad

Abstract: We study the space of functions computed by random-layered machines, including deep neural networks and Boolean circuits. Investigating the distribution of Boolean functions computed on the recurrent and layer-dependent architectures, we find that it is the same in both models. Depending on the initial conditions and computing elements used, we characterize the space of functions computed at the l… ▽ More We study the space of functions computed by random-layered machines, including deep neural networks and Boolean circuits. Investigating the distribution of Boolean functions computed on the recurrent and layer-dependent architectures, we find that it is the same in both models. Depending on the initial conditions and computing elements used, we characterize the space of functions computed at the large depth limit and show that the macroscopic entropy of Boolean functions is either monotonically increasing or decreasing with the growing depth. △ Less

Submitted 13 October, 2020; v1 submitted 19 April, 2020; originally announced April 2020.

Journal ref: Phys. Rev. Lett. 125, 168301 (2020)

Scalable Influence Estimation Without Sampling

Authors: Andrey Y. Lokhov, David Saad

Abstract: In a diffusion process on a network, how many nodes are expected to be influenced by a set of initial spreaders? This natural problem, often referred to as influence estimation, boils down to computing the marginal probability that a given node is active at a given time when the process starts from specified initial condition. Among many other applications, this task is crucial for a well-studied… ▽ More In a diffusion process on a network, how many nodes are expected to be influenced by a set of initial spreaders? This natural problem, often referred to as influence estimation, boils down to computing the marginal probability that a given node is active at a given time when the process starts from specified initial condition. Among many other applications, this task is crucial for a well-studied problem of influence maximization: finding optimal spreaders in a social network that maximize the influence spread by a certain time horizon. Indeed, influence estimation needs to be called multiple times for comparing candidate seed sets. Unfortunately, in many models of interest an exact computation of marginals is #P-hard. In practice, influence is often estimated using Monte-Carlo sampling methods that require a large number of runs for obtaining a high-fidelity prediction, especially at large times. It is thus desirable to develop analytic techniques as an alternative to sampling methods. Here, we suggest an algorithm for estimating the influence function in popular independent cascade model based on a scalable dynamic message-passing approach. This method has a computational complexity of a single Monte-Carlo simulation and provides an upper bound on the expected spread on a general graph, yielding exact answer for treelike networks. We also provide dynamic message-passing equations for a stochastic version of the linear threshold model. The resulting saving of a potentially large sampling factor in the running time compared to simulation-based techniques hence makes it possible to address large-scale problem instances. △ Less

Submitted 29 December, 2019; originally announced December 2019.

Large Deviation Analysis of Function Sensitivity in Random Deep Neural Networks

Authors: Bo Li, David Saad

Abstract: Mean field theory has been successfully used to analyze deep neural networks (DNN) in the infinite size limit. Given the finite size of realistic DNN, we utilize the large deviation theory and path integral analysis to study the deviation of functions represented by DNN from their typical mean field solutions. The parameter perturbations investigated include weight sparsification (dilution) and bi… ▽ More Mean field theory has been successfully used to analyze deep neural networks (DNN) in the infinite size limit. Given the finite size of realistic DNN, we utilize the large deviation theory and path integral analysis to study the deviation of functions represented by DNN from their typical mean field solutions. The parameter perturbations investigated include weight sparsification (dilution) and binarization, which are commonly used in model simplification, for both ReLU and sign activation functions. We find that random networks with ReLU activation are more robust to parameter perturbations with respect to their counterparts with sign activation, which arguably is reflected in the simplicity of the functions they generate. △ Less

Submitted 20 February, 2020; v1 submitted 13 October, 2019; originally announced October 2019.

Journal ref: J. Phys. A: Math. Theor. 53. 104002 (2020)

Self-Organization Scheme for Balanced Routing in Large-Scale Multi-Hop Networks

Authors: Mihai-Alin Badiu, David Saad, Justin P. Coon

Abstract: We propose a self-organization scheme for cost-effective and load-balanced routing in multi-hop networks. To avoid overloading nodes that provide favourable routing conditions, we assign each node with a cost function that penalizes high loads. Thus, finding routes to sink nodes is formulated as an optimization problem in which the global objective function strikes a balance between route costs an… ▽ More We propose a self-organization scheme for cost-effective and load-balanced routing in multi-hop networks. To avoid overloading nodes that provide favourable routing conditions, we assign each node with a cost function that penalizes high loads. Thus, finding routes to sink nodes is formulated as an optimization problem in which the global objective function strikes a balance between route costs and node loads. We apply belief propagation (its min-sum version) to solve the network optimization problem and obtain a distributed algorithm whereby the nodes collectively discover globally optimal routes by performing low-complexity computations and exchanging messages with their neighbours. We prove that the proposed method converges to the global optimum after a finite number of local exchanges of messages. Finally, we demonstrate numerically our framework's efficacy in balancing the node loads and study the trade-off between load reduction and total cost minimization. △ Less

Submitted 10 August, 2018; originally announced August 2018.

Exploring the Function Space of Deep-Learning Machines

Authors: Bo Li, David Saad

Abstract: The function space of deep-learning machines is investigated by studying growth in the entropy of functions of a given error with respect to a reference function, realized by a deep-learning machine. Using physics-inspired methods we study both sparsely and densely-connected architectures to discover a layer-wise convergence of candidate functions, marked by a corresponding reduction in entropy wh… ▽ More The function space of deep-learning machines is investigated by studying growth in the entropy of functions of a given error with respect to a reference function, realized by a deep-learning machine. Using physics-inspired methods we study both sparsely and densely-connected architectures to discover a layer-wise convergence of candidate functions, marked by a corresponding reduction in entropy when approaching the reference function, gain insight into the importance of having a large number of layers, and observe phase transitions as the error increases. △ Less

Submitted 9 August, 2018; v1 submitted 4 August, 2017; originally announced August 2017.

Comments: New examples of networks with ReLU activation and convolutional networks are included

Journal ref: Phys. Rev. Lett. 120, 248301 (2018)

Optimal Deployment of Resources for Maximizing Impact in Spreading Processes

Authors: Andrey Y. Lokhov, David Saad

Abstract: The effective use of limited resources for controlling spreading processes on networks is of prime significance in diverse contexts, ranging from the identification of "influential spreaders" for maximizing information dissemination and targeted interventions in regulatory networks, to the development of mitigation policies for infectious diseases and financial contagion in economic systems. Solut… ▽ More The effective use of limited resources for controlling spreading processes on networks is of prime significance in diverse contexts, ranging from the identification of "influential spreaders" for maximizing information dissemination and targeted interventions in regulatory networks, to the development of mitigation policies for infectious diseases and financial contagion in economic systems. Solutions for these optimization tasks that are based purely on topological arguments are not fully satisfactory; in realistic settings the problem is often characterized by heterogeneous interactions and requires interventions over a finite time window via a restricted set of controllable nodes. The optimal distribution of available resources hence results from an interplay between network topology and spreading dynamics. We show how these problems can be addressed as particular instances of a universal analytical framework based on a scalable dynamic message-passing approach and demonstrate the efficacy of the method on a variety of real-world examples. △ Less

Submitted 29 August, 2016; originally announced August 2016.

Comments: 10 pages and appendices, 6 figures

Shortest node-disjoint paths on random graphs

Authors: Caterina De Bacco, Silvio Franz, David Saad, Chi Ho Yeung

Abstract: A localized method to distribute paths on random graphs is devised, aimed at finding the shortest paths between given source/destination pairs while avoiding path overlaps at nodes. We propose a method based on message-passing techniques to process global information and distribute paths optimally. Statistical properties such as scaling with system size and number of paths, average path-length and… ▽ More A localized method to distribute paths on random graphs is devised, aimed at finding the shortest paths between given source/destination pairs while avoiding path overlaps at nodes. We propose a method based on message-passing techniques to process global information and distribute paths optimally. Statistical properties such as scaling with system size and number of paths, average path-length and the transition to the frustrated regime are analysed. The performance of the suggested algorithm is evaluated through a comparison against a greedy algorithm. △ Less

Submitted 18 May, 2014; v1 submitted 31 January, 2014; originally announced January 2014.

Comments: 22 pages, 14 figures

Journal ref: J. Stat. Mech. (2014) P07009

Message-Passing Algorithms for Optimal Utilization of Cognitive Radio Networks

Authors: Hamed Mahmoudi, Georgios Rodolakis, Leonidas Georgiadis, David Saad

Abstract: Cognitive Radio has been proposed as a key technology to significantly improve spectrum usage in wireless networks by enabling unlicensed users to access unused resource. We present new algorithms that are needed for the implementation of opportunistic scheduling policies that maximize the throughput utilization of resources by secondary users, under maximum interference constraints imposed by exi… ▽ More Cognitive Radio has been proposed as a key technology to significantly improve spectrum usage in wireless networks by enabling unlicensed users to access unused resource. We present new algorithms that are needed for the implementation of opportunistic scheduling policies that maximize the throughput utilization of resources by secondary users, under maximum interference constraints imposed by existing primary users. Our approach is based on the Belief Propagation (BP) algorithm, which is advantageous due to its simplicity and potential for distributed implementation. We examine convergence properties and evaluate the performance of the proposed BP algorithms via simulations and demonstrate that the results compare favorably with a benchmark greedy strategy. △ Less

Submitted 21 October, 2013; originally announced October 2013.

From the Physics of Interacting Polymers to Optimizing Routes on the London Underground

Authors: Chi Ho Yeung, David Saad, K. Y. Michael Wong

Abstract: Optimizing paths on networks is crucial for many applications, from subway traffic to Internet communication. As global path optimization that takes account of all path-choices simultaneously is computationally hard, most existing routing algorithms optimize paths individually, thus providing sub-optimal solutions. We employ the physics of interacting polymers and disordered systems to analyze mac… ▽ More Optimizing paths on networks is crucial for many applications, from subway traffic to Internet communication. As global path optimization that takes account of all path-choices simultaneously is computationally hard, most existing routing algorithms optimize paths individually, thus providing sub-optimal solutions. We employ the physics of interacting polymers and disordered systems to analyze macroscopic properties of generic path-optimization problems and derive a simple, principled, generic and distributed routing algorithm capable of considering simultaneously all individual path choices. We demonstrate the efficacy of the new algorithm by applying it to: (i) random graphs resembling Internet overlay networks; (ii) travel on the London underground network based on Oyster-card data; and (iii) the global airport network. Analytically derived macroscopic properties give rise to insightful new routing phenomena, including phase transitions and scaling laws, which facilitate better understanding of the appropriate operational regimes and their limitations that are difficult to obtain otherwise. △ Less

Submitted 3 September, 2013; originally announced September 2013.

Comments: 6 pages, 6 figures. Supplementary information available at: http://www.pnas.org/content/suppl/2013/07/29/1301111110.DCSupplemental

Journal ref: PNAS Vol. 110, No. 34, P. 13717-13722 (August 20, 2013)

The Competition for Shortest Paths on Sparse Graphs

Authors: Chi Ho Yeung, David Saad

Abstract: Optimal paths connecting randomly selected network nodes and fixed routers are studied analytically in the presence of non-linear overlap cost that penalizes congestion. Routing becomes increasingly more difficult as the number of selected nodes increases and exhibits ergodicity breaking in the case of multiple routers. A distributed linearly-scalable routing algorithm is devised. The ground state… ▽ More Optimal paths connecting randomly selected network nodes and fixed routers are studied analytically in the presence of non-linear overlap cost that penalizes congestion. Routing becomes increasingly more difficult as the number of selected nodes increases and exhibits ergodicity breaking in the case of multiple routers. A distributed linearly-scalable routing algorithm is devised. The ground state of such systems reveals non-monotonic complex behaviors in both average path-length and algorithmic convergence, depending on the network topology, and densities of communicating nodes and routers. △ Less

Submitted 1 February, 2012; originally announced February 2012.

Comments: 4 pages, 4 figures

Journal ref: Phys. Rev. Lett. 108, 208701 (2012)

Networking - A Statistical Physics Perspective

Authors: Chi Ho Yeung, David Saad

Abstract: Efficient networking has a substantial economic and societal impact in a broad range of areas including transportation systems, wired and wireless communications and a range of Internet applications. As transportation and communication networks become increasingly more complex, the ever increasing demand for congestion control, higher traffic capacity, quality of service, robustness and reduced en… ▽ More Efficient networking has a substantial economic and societal impact in a broad range of areas including transportation systems, wired and wireless communications and a range of Internet applications. As transportation and communication networks become increasingly more complex, the ever increasing demand for congestion control, higher traffic capacity, quality of service, robustness and reduced energy consumption require new tools and methods to meet these conflicting requirements. The new methodology should serve for gaining better understanding of the properties of networking systems at the macroscopic level, as well as for the development of new principled optimization and management algorithms at the microscopic level. Methods of statistical physics seem best placed to provide new approaches as they have been developed specifically to deal with non-linear large scale systems. This paper aims at presenting an overview of tools and methods that have been developed within the statistical physics community and that can be readily applied to address the emerging problems in networking. These include diffusion processes, methods from disordered systems and polymer physics, probabilistic inference, which have direct relevance to network routing, file and frequency distribution, the exploration of network structures and vulnerability, and various other practical networking applications. △ Less

Submitted 2 October, 2012; v1 submitted 13 October, 2011; originally announced October 2011.

Comments: (Review article) 71 pages, 14 figures

Journal ref: J. Phys. A: Math. Theor., Vol. 46, P. 103001 (2013)

The interplay of microscopic and mesoscopic structure in complex networks

Authors: Joerg Reichardt, Roberto Alamino, David Saad

Abstract: Not all nodes in a network are created equal. Differences and similarities exist at both individual node and group levels. Disentangling single node from group properties is crucial for network modeling and structural inference. Based on unbiased generative probabilistic exponential random graph models and employing distributive message passing techniques, we present an efficient algorithm that al… ▽ More Not all nodes in a network are created equal. Differences and similarities exist at both individual node and group levels. Disentangling single node from group properties is crucial for network modeling and structural inference. Based on unbiased generative probabilistic exponential random graph models and employing distributive message passing techniques, we present an efficient algorithm that allows one to separate the contributions of individual nodes and groups of nodes to the network structure. This leads to improved detection accuracy of latent class structure in real world data sets compared to models that focus on group structure alone. Furthermore, the inclusion of hitherto neglected group specific effects in models used to assess the statistical significance of small subgraph (motif) distributions in networks may be sufficient to explain most of the observed statistics. We show the predictive power of such generative models in forecasting putative gene-disease associations in the Online Mendelian Inheritance in Man (OMIM) database. The approach is suitable for both directed and undirected uni-partite as well as for bipartite networks. △ Less

Submitted 20 December, 2010; originally announced December 2010.

Computing with Noise - Phase Transitions in Boolean Formulas

Authors: Alexander Mozeika, David Saad, Jack Raymond

Abstract: Computing circuits composed of noisy logical gates and their ability to represent arbitrary Boolean functions with a given level of error are investigated within a statistical mechanics setting. Bounds on their performance, derived in the information theory literature for specific gates, are straightforwardly retrieved, generalized and identified as the corresponding typical-case phase transitio… ▽ More Computing circuits composed of noisy logical gates and their ability to represent arbitrary Boolean functions with a given level of error are investigated within a statistical mechanics setting. Bounds on their performance, derived in the information theory literature for specific gates, are straightforwardly retrieved, generalized and identified as the corresponding typical-case phase transitions. This framework paves the way for obtaining new results on error-rates, function-depth and sensitivity, and their dependence on the gate-type and noise model used. △ Less

Submitted 27 August, 2009; originally announced August 2009.

Comments: 10 pages, 2 figures

Composite CDMA - A statistical mechanics analysis

Authors: Jack Raymond, David Saad

Abstract: Code Division Multiple Access (CDMA) in which the spreading code assignment to users contains a random element has recently become a cornerstone of CDMA research. The random element in the construction is particular attractive as it provides robustness and flexibility in utilising multi-access channels, whilst not making significant sacrifices in terms of transmission power. Random codes are gen… ▽ More Code Division Multiple Access (CDMA) in which the spreading code assignment to users contains a random element has recently become a cornerstone of CDMA research. The random element in the construction is particular attractive as it provides robustness and flexibility in utilising multi-access channels, whilst not making significant sacrifices in terms of transmission power. Random codes are generated from some ensemble, here we consider the possibility of combining two standard paradigms, sparsely and densely spread codes, in a single composite code ensemble. The composite code analysis includes a replica symmetric calculation of performance in the large system limit, and investigation of finite systems through a composite belief propagation algorithm. A variety of codes are examined with a focus on the high multi-access interference regime. In both the large size limit and finite systems we demonstrate scenarios in which the composite code has typical performance exceeding sparse and dense codes at equivalent signal to noise ratio. △ Less

Submitted 22 May, 2009; v1 submitted 14 November, 2008; originally announced November 2008.

Comments: 23 pages, 11 figures, Sigma Phi 2008 conference submission - submitted to J.Stat.Mech

Journal ref: J. Stat. Mech. (2009) P05015

Randomness and metastability in CDMA paradigms

Authors: Jack Raymond, David Saad

Abstract: Code Division Multiple Access (CDMA) in which the signature code assignment to users contains a random element has recently become a cornerstone of CDMA research. The random element in the construction is particularly attractive in that it provides robustness and flexibility in application, whilst not making significant sacrifices in terms of multiuser efficiency. We present results for sparse r… ▽ More Code Division Multiple Access (CDMA) in which the signature code assignment to users contains a random element has recently become a cornerstone of CDMA research. The random element in the construction is particularly attractive in that it provides robustness and flexibility in application, whilst not making significant sacrifices in terms of multiuser efficiency. We present results for sparse random codes of two types, with and without modulation. Simple microscopic consideration on system samples would suggest differences in the phase space of the two models, but we demonstrate that the thermodynamic results and metastable states are equivalent in the minimum bit error rate detector. We analyse marginal properties of interactions and also make analogies to constraint satisfiability problems in order to understand qualitative features of detection and metastable states. This may have consequences for developing algorithmic methods to escape metastable states, thus improving decoding performance. △ Less

Submitted 23 June, 2008; v1 submitted 28 November, 2007; originally announced November 2007.

Comments: 5 pages, 3 figures, prepared IEEE.cls, accepted physcomnet-08

Minimizing Unsatisfaction in Colourful Neighbourhoods

Authors: K. Y. Michael Wong, David Saad

Abstract: Colouring sparse graphs under various restrictions is a theoretical problem of significant practical relevance. Here we consider the problem of maximizing the number of different colours available at the nodes and their neighbourhoods, given a predetermined number of colours. In the analytical framework of a tree approximation, carried out at both zero and finite temperatures, solutions obtained… ▽ More Colouring sparse graphs under various restrictions is a theoretical problem of significant practical relevance. Here we consider the problem of maximizing the number of different colours available at the nodes and their neighbourhoods, given a predetermined number of colours. In the analytical framework of a tree approximation, carried out at both zero and finite temperatures, solutions obtained by population dynamics give rise to estimates of the threshold connectivity for the incomplete to complete transition, which are consistent with those of existing algorithms. The nature of the transition as well as the validity of the tree approximation are investigated. △ Less

Submitted 1 July, 2008; v1 submitted 29 April, 2007; originally announced April 2007.

Comments: 28 pages, 12 figures, substantially revised with additional explanation

Journal ref: J. Phys. A: Math. Theor. 41, 324023 (2008).

Sparsely-spread CDMA - a statistical mechanics based analysis

Authors: Jack Raymond, David Saad

Abstract: Sparse Code Division Multiple Access (CDMA), a variation on the standard CDMA method in which the spreading (signature) matrix contains only a relatively small number of non-zero elements, is presented and analysed using methods of statistical physics. The analysis provides results on the performance of maximum likelihood decoding for sparse spreading codes in the large system limit. We present… ▽ More Sparse Code Division Multiple Access (CDMA), a variation on the standard CDMA method in which the spreading (signature) matrix contains only a relatively small number of non-zero elements, is presented and analysed using methods of statistical physics. The analysis provides results on the performance of maximum likelihood decoding for sparse spreading codes in the large system limit. We present results for both cases of regular and irregular spreading matrices for the binary additive white Gaussian noise channel (BIAWGN) with a comparison to the canonical (dense) random spreading code. △ Less

Submitted 30 April, 2008; v1 submitted 1 April, 2007; originally announced April 2007.

Comments: 23 pages, 5 figures, figure 1 amended since published version

Journal ref: J. Phys. A: Math. Theor. 40 No 41 (12 October 2007) 12315-12333

Improved message passing for inference in densely connected systems

Authors: Juan P. Neirotti, David Saad

Abstract: An improved inference method for densely connected systems is presented. The approach is based on passing condensed messages between variables, representing macroscopic averages of microscopic messages. We extend previous work that showed promising results in cases where the solution space is contiguous to cases where fragmentation occurs. We apply the method to the signal detection problem of C… ▽ More An improved inference method for densely connected systems is presented. The approach is based on passing condensed messages between variables, representing macroscopic averages of microscopic messages. We extend previous work that showed promising results in cases where the solution space is contiguous to cases where fragmentation occurs. We apply the method to the signal detection problem of Code Division Multiple Access (CDMA) for demonstrating its potential. A highly efficient practical algorithm is also derived on the basis of insight gained from the analysis. △ Less

Submitted 24 March, 2005; originally announced March 2005.