Showing 1–15 of 15 results for author: Sondak, D

Extending a Physics-Informed Machine Learning Network for Superresolution Studies of Rayleigh-Bénard Convection

Authors: Diane M. Salim, Blakesley Burkhart, David Sondak

Abstract: Advancing our understanding of astrophysical turbulence is bottlenecked by the limited resolution of numerical simulations that may not fully sample scales in the inertial range. Machine learning (ML) techniques have demonstrated promise in up-scaling resolution in both image analysis and numerical simulations (i.e., superresolution). Here we employ and further develop a physics-constrained convol… ▽ More Advancing our understanding of astrophysical turbulence is bottlenecked by the limited resolution of numerical simulations that may not fully sample scales in the inertial range. Machine learning (ML) techniques have demonstrated promise in up-scaling resolution in both image analysis and numerical simulations (i.e., superresolution). Here we employ and further develop a physics-constrained convolutional neural network (CNN) ML model called "MeshFreeFlowNet'' (MFFN) for superresolution studies of turbulent systems. The model is trained both on the simulation images as well as the evaluated PDEs, making it sensitive to the underlying physics of a particular fluid system. We develop a framework for 2D turbulent Rayleigh-Bénard convection (RBC) generated with the \textsc{Dedalus} code by modifying the MFFN architecture to include the full set of simulation PDEs and the boundary conditions. Our training set includes fully developed turbulence sampling Rayleigh numbers ($Ra$) of $Ra=10^6-10^{10}$. We evaluate the success of the learned simulations by comparing the power spectra of the direct \textsc{Dedalus} simulation to the predicted model output, and compare both ground truth and predicted power spectral inertial range scalings to theoretical predictions. We find that the updated network performs well at all $Ra$ studied here in recovering large-scale information, including the inertial range slopes. The superresolution prediction is overly dissipative at smaller scales than that of the inertial range in all cases, but the smaller-scales are better recovered in more turbulent, than laminar, regimes. This is likely because more turbulent systems have a rich variety of structures at many length scales compared to laminar flows. △ Less

Submitted 31 January, 2024; v1 submitted 5 July, 2023; originally announced July 2023.

Comments: 11 pages, 10 figures

Multi-Task Learning based Convolutional Models with Curriculum Learning for the Anisotropic Reynolds Stress Tensor in Turbulent Duct Flow

Authors: Haitz Sáez de Ocáriz Borde, David Sondak, Pavlos Protopapas

Abstract: The Reynolds-averaged Navier-Stokes (RANS) equations require accurate modeling of the anisotropic Reynolds stress tensor. Traditional closure models, while sophisticated, often only apply to restricted flow configurations. Researchers have started using machine learning approaches to tackle this problem by developing more general closure models informed by data. In this work we build upon recent c… ▽ More The Reynolds-averaged Navier-Stokes (RANS) equations require accurate modeling of the anisotropic Reynolds stress tensor. Traditional closure models, while sophisticated, often only apply to restricted flow configurations. Researchers have started using machine learning approaches to tackle this problem by developing more general closure models informed by data. In this work we build upon recent convolutional neural network architectures used for turbulence modeling and propose a multi-task learning-based fully convolutional neural network that is able to accurately predict the normalized anisotropic Reynolds stress tensor for turbulent duct flows. Furthermore, we also explore the application of curriculum learning to data-driven turbulence modeling. △ Less

Submitted 31 January, 2022; v1 submitted 30 October, 2021; originally announced November 2021.

Port-Hamiltonian Neural Networks for Learning Explicit Time-Dependent Dynamical Systems

Authors: Shaan Desai, Marios Mattheakis, David Sondak, Pavlos Protopapas, Stephen Roberts

Abstract: Accurately learning the temporal behavior of dynamical systems requires models with well-chosen learning biases. Recent innovations embed the Hamiltonian and Lagrangian formalisms into neural networks and demonstrate a significant improvement over other approaches in predicting trajectories of physical systems. These methods generally tackle autonomous systems that depend implicitly on time or sys… ▽ More Accurately learning the temporal behavior of dynamical systems requires models with well-chosen learning biases. Recent innovations embed the Hamiltonian and Lagrangian formalisms into neural networks and demonstrate a significant improvement over other approaches in predicting trajectories of physical systems. These methods generally tackle autonomous systems that depend implicitly on time or systems for which a control signal is known apriori. Despite this success, many real world dynamical systems are non-autonomous, driven by time-dependent forces and experience energy dissipation. In this study, we address the challenge of learning from such non-autonomous systems by embedding the port-Hamiltonian formalism into neural networks, a versatile framework that can capture energy dissipation and time-dependent control forces. We show that the proposed \emph{port-Hamiltonian neural network} can efficiently learn the dynamics of nonlinear physical systems of practical interest and accurately recover the underlying stationary Hamiltonian, time-dependent force, and dissipative coefficient. A promising outcome of our network is its ability to learn and predict chaotic systems such as the Duffing equation, for which the trajectories are typically hard to learn. △ Less

Submitted 16 July, 2021; originally announced July 2021.

Comments: [under review]

Journal ref: Phys. Rev. E 104, 034312 (2021)

Convolutional Neural Network Models and Interpretability for the Anisotropic Reynolds Stress Tensor in Turbulent One-dimensional Flows

Authors: Haitz Sáez de Ocáriz Borde, David Sondak, Pavlos Protopapas

Abstract: The Reynolds-averaged Navier-Stokes (RANS) equations are widely used in turbulence applications. They require accurately modeling the anisotropic Reynolds stress tensor, for which traditional Reynolds stress closure models only yield reliable results in some flow configurations. In the last few years, there has been a surge of work aiming at using data-driven approaches to tackle this problem. The… ▽ More The Reynolds-averaged Navier-Stokes (RANS) equations are widely used in turbulence applications. They require accurately modeling the anisotropic Reynolds stress tensor, for which traditional Reynolds stress closure models only yield reliable results in some flow configurations. In the last few years, there has been a surge of work aiming at using data-driven approaches to tackle this problem. The majority of previous work has focused on the development of fully-connected networks for modeling the anisotropic Reynolds stress tensor. In this paper, we expand upon recent work for turbulent channel flow and develop new convolutional neural network (CNN) models that are able to accurately predict the normalized anisotropic Reynolds stress tensor. We apply the new CNN model to a number of one-dimensional turbulent flows. Additionally, we present interpretability techniques that help drive the model design and provide guidance on the model behavior in relation to the underlying physics. △ Less

Submitted 29 June, 2021; originally announced June 2021.

Learning a Reduced Basis of Dynamical Systems using an Autoencoder

Authors: David Sondak, Pavlos Protopapas

Abstract: Machine learning models have emerged as powerful tools in physics and engineering. Although flexible, a fundamental challenge remains on how to connect new machine learning models with known physics. In this work, we present an autoencoder with latent space penalization, which discovers finite dimensional manifolds underlying the partial differential equations of physics. We test this method on th… ▽ More Machine learning models have emerged as powerful tools in physics and engineering. Although flexible, a fundamental challenge remains on how to connect new machine learning models with known physics. In this work, we present an autoencoder with latent space penalization, which discovers finite dimensional manifolds underlying the partial differential equations of physics. We test this method on the Kuramoto-Sivashinsky (K-S), Korteweg-de Vries (KdV), and damped KdV equations. We show that the resulting optimal latent space of the K-S equation is consistent with the dimension of the inertial manifold. The results for the KdV equation imply that there is no reduced latent space, which is consistent with the truly infinite dimensional dynamics of the KdV equation. In the case of the damped KdV equation, we find that the number of active dimensions decreases with increasing damping coefficient. We then uncover a nonlinear basis representing the manifold of the latent space for the K-S equation. △ Less

Submitted 14 November, 2020; originally announced November 2020.

Comments: 9 pages, 9 figures, two tables

Journal ref: Phys. Rev. E 104, 034202 (2021)

Solving Differential Equations Using Neural Network Solution Bundles

Authors: Cedric Flamant, Pavlos Protopapas, David Sondak

Abstract: The time evolution of dynamical systems is frequently described by ordinary differential equations (ODEs), which must be solved for given initial conditions. Most standard approaches numerically integrate ODEs producing a single solution whose values are computed at discrete times. When many varied solutions with different initial conditions to the ODE are required, the computational cost can beco… ▽ More The time evolution of dynamical systems is frequently described by ordinary differential equations (ODEs), which must be solved for given initial conditions. Most standard approaches numerically integrate ODEs producing a single solution whose values are computed at discrete times. When many varied solutions with different initial conditions to the ODE are required, the computational cost can become significant. We propose that a neural network be used as a solution bundle, a collection of solutions to an ODE for various initial states and system parameters. The neural network solution bundle is trained with an unsupervised loss that does not require any prior knowledge of the sought solutions, and the resulting object is differentiable in initial conditions and system parameters. The solution bundle exhibits fast, parallelizable evaluation of the system state, facilitating the use of Bayesian inference for parameter estimation in real dynamical systems. △ Less

Submitted 16 June, 2020; originally announced June 2020.

Comments: 21 pages, 12 figures, 10 tables

Coherent Solutions and Transition to Turbulence in Two-Dimensional Rayleigh-Bénard Convection

Authors: Parvathi Kooloth, David Sondak, Leslie M. Smith

Abstract: For two-dimensional Rayleigh-Bénard convection, classes of unstable, steady solutions were previously computed using numerical continuation (Waleffe, 2015; Sondak, 2015). The `primary' steady solution bifurcates from the conduction state at $Ra \approx 1708$, and has a characteristic aspect ratio (length/height) of approximately $2$. The primary solution corresponds to one pair of counterclockwise… ▽ More For two-dimensional Rayleigh-Bénard convection, classes of unstable, steady solutions were previously computed using numerical continuation (Waleffe, 2015; Sondak, 2015). The `primary' steady solution bifurcates from the conduction state at $Ra \approx 1708$, and has a characteristic aspect ratio (length/height) of approximately $2$. The primary solution corresponds to one pair of counterclockwise-clockwise convection rolls with a temperature updraft in between and an adjacent downdraft on the sides. By adjusting the horizontal length of the domain, (Waleffe, 2015; Sondak, 2015) also found steady, maximal heat transport solutions, with characteristic aspect ratio less than $2$ and decreasing with increasing $Ra$. Compared to the primary solutions, optimal heat transport solutions have modifications to boundary layer thickness, the horizontal length scale of the plume, and the structure of the downdrafts. The current study establishes a direct link between these (unstable) steady solutions and transition to turbulence for $Pr = 7$ and $Pr = 100$. For transitional values of $Ra$, the primary and optimal heat transport solutions both appear prominently in appropriately-sized sub-fields of the time-evolving temperature fields. For $Ra$ beyond transitional, our data analysis shows persistence of the primary solution for $Pr = 7$, while the optimal heat transport solutions are more easily detectable for $Pr = 100$. In both cases $Pr = 7$ and $Pr = 100$, the relative prevalence of primary and optimal solutions is consistent with the $Nu$ vs. $Ra$ scalings for the numerical data and the steady solutions. △ Less

Submitted 24 June, 2020; originally announced June 2020.

Comments: 34 pages, 23 figures

Journal ref: Phys. Rev. Fluids 6, 013501 (2021)

High Rayleigh number variational multiscale large eddy simulations of Rayleigh-Bénard Convection

Authors: David Sondak, Thomas M. Smith, Roger P. Pawlowski, Sidafa Conde, John N. Shadid

Abstract: The variational multiscale (VMS) formulation is used to develop residual-based VMS large eddy simulation (LES) models for Rayleigh-Bénard convection. The resulting model is a mixed model that incorporates the VMS model and an eddy viscosity model. The Wall-Adapting Local Eddy-viscosity (WALE) model is used as the eddy viscosity model in this work. The new LES models were implemented in the finite… ▽ More The variational multiscale (VMS) formulation is used to develop residual-based VMS large eddy simulation (LES) models for Rayleigh-Bénard convection. The resulting model is a mixed model that incorporates the VMS model and an eddy viscosity model. The Wall-Adapting Local Eddy-viscosity (WALE) model is used as the eddy viscosity model in this work. The new LES models were implemented in the finite element code Drekar. Simulations are performed using continuous, piecewise linear finite elements. The simulations ranged from $Ra = 10^6$ to $Ra = 10^{14}$ and were conducted at $Pr = 1$ and $Pr = 7$. Two domains were considered: a two-dimensional domain of aspect ratio 2 with a fluid confined between two parallel plates and a three-dimensional cylinder of aspect ratio $1/4$. The Nusselt number from the VMS results is compared against three dimensional direct numerical simulations and experiments. In all cases, the VMS results are in good agreement with existing literature. △ Less

Submitted 20 May, 2020; originally announced May 2020.

Comments: 9 pages, 6 figures. Submitted to Mechanics Research Communications

Hamiltonian neural networks for solving equations of motion

Authors: Marios Mattheakis, David Sondak, Akshunna S. Dogra, Pavlos Protopapas

Abstract: There has been a wave of interest in applying machine learning to study dynamical systems. We present a Hamiltonian neural network that solves the differential equations that govern dynamical systems. This is an equation-driven machine learning method where the optimization process of the network depends solely on the predicted functions without using any ground truth data. The model learns soluti… ▽ More There has been a wave of interest in applying machine learning to study dynamical systems. We present a Hamiltonian neural network that solves the differential equations that govern dynamical systems. This is an equation-driven machine learning method where the optimization process of the network depends solely on the predicted functions without using any ground truth data. The model learns solutions that satisfy, up to an arbitrarily small error, Hamilton's equations and, therefore, conserve the Hamiltonian invariants. The choice of an appropriate activation function drastically improves the predictability of the network. Moreover, an error analysis is derived and states that the numerical errors depend on the overall network performance. The Hamiltonian network is then employed to solve the equations for the nonlinear oscillator and the chaotic Henon-Heiles dynamical system. In both systems, a symplectic Euler integrator requires two orders more evaluation points than the Hamiltonian network in order to achieve the same order of the numerical error in the predicted phase space trajectories. △ Less

Submitted 26 April, 2022; v1 submitted 29 January, 2020; originally announced January 2020.

Journal ref: Phys. Rev. E 105, 065305 (2022)

Neural Network Models for the Anisotropic Reynolds Stress Tensor in Turbulent Channel Flow

Authors: Rui Fang, David Sondak, Pavlos Protopapas, Sauro Succi

Abstract: Reynolds-averaged Navier-Stokes (RANS) equations are presently one of the most popular models for simulating turbulence. Performing RANS simulation requires additional modeling for the anisotropic Reynolds stress tensor, but traditional Reynolds stress closure models lead to only partially reliable predictions. Recently, data-driven turbulence models for the Reynolds anisotropy tensor involving no… ▽ More Reynolds-averaged Navier-Stokes (RANS) equations are presently one of the most popular models for simulating turbulence. Performing RANS simulation requires additional modeling for the anisotropic Reynolds stress tensor, but traditional Reynolds stress closure models lead to only partially reliable predictions. Recently, data-driven turbulence models for the Reynolds anisotropy tensor involving novel machine learning techniques have garnered considerable attention and have been rapidly developed. Focusing on modeling the Reynolds stress closure for the specific case of turbulent channel flow, this paper proposes three modifications to a standard neural network to account for the no-slip boundary condition of the anisotropy tensor, the Reynolds number dependence, and spatial non-locality. The modified models are shown to provide increased predicative accuracy compared to the standard neural network when they are trained and tested on channel flow at different Reynolds numbers. The best performance is yielded by the model combining the boundary condition enforcement and Reynolds number injection. This model also outperforms the Tensor Basis Neural Network (Ling et al., 2016) on the turbulent channel flow dataset. △ Less

Submitted 8 September, 2019; originally announced September 2019.

Physical Symmetries Embedded in Neural Networks

Authors: M. Mattheakis, P. Protopapas, D. Sondak, M. Di Giovanni, E. Kaxiras

Abstract: Neural networks are a central technique in machine learning. Recent years have seen a wave of interest in applying neural networks to physical systems for which the governing dynamics are known and expressed through differential equations. Two fundamental challenges facing the development of neural networks in physics applications is their lack of interpretability and their physics-agnostic design… ▽ More Neural networks are a central technique in machine learning. Recent years have seen a wave of interest in applying neural networks to physical systems for which the governing dynamics are known and expressed through differential equations. Two fundamental challenges facing the development of neural networks in physics applications is their lack of interpretability and their physics-agnostic design. The focus of the present work is to embed physical constraints into the structure of the neural network to address the second fundamental challenge. By constraining tunable parameters (such as weights and biases) and adding special layers to the network, the desired constraints are guaranteed to be satisfied without the need for explicit regularization terms. This is demonstrated on upervised and unsupervised networks for two basic symmetries: even/odd symmetry of a function and energy conservation. In the supervised case, the network with embedded constraints is shown to perform well on regression problems while simultaneously obeying the desired constraints whereas a traditional network fits the data but violates the underlying constraints. Finally, a new unsupervised neural network is proposed that guarantees energy conservation through an embedded symplectic structure. The symplectic neural network is used to solve a system of energy-conserving differential equations and out-performs an unsupervised, non-symplectic neural network. △ Less

Submitted 29 January, 2020; v1 submitted 18 April, 2019; originally announced April 2019.

Comments: This is the same manuscript with version 1 (arXiv:1904.08991v1) which accidentally was replaced 16 pages, 8 figures

Deep learning for turbulent channel flow

Authors: Rui Fang, David Sondak, Pavlos Protopapas, Sauro Succi

Abstract: Turbulence modeling is a classical approach to address the multiscale nature of fluid turbulence. Instead of resolving all scales of motion, which is currently mathematically and numerically intractable, reduced models that capture the large-scale behavior are derived. One of the most popular reduced models is the Reynolds averaged Navier-Stokes (RANS) equations. The goal is to solve the RANS equa… ▽ More Turbulence modeling is a classical approach to address the multiscale nature of fluid turbulence. Instead of resolving all scales of motion, which is currently mathematically and numerically intractable, reduced models that capture the large-scale behavior are derived. One of the most popular reduced models is the Reynolds averaged Navier-Stokes (RANS) equations. The goal is to solve the RANS equations for the mean velocity and pressure field. However, the RANS equations contain a term called the Reynolds stress tensor, which is not known in terms of the mean velocity field. Many RANS turbulence models have been proposed to model the Reynolds stress tensor in terms of the mean velocity field, but are usually not suitably general for all flow fields of interest. Data-driven turbulence models have recently garnered considerable attention and have been rapidly developed. In a seminal work, Ling et al (2016) developed the tensor basis neural network (TBNN), which was used to learn a general Galilean invariant model for the Reynolds stress tensor. The TBNN was applied to a variety of flow fields with encouraging results. In the present study, the TBNN is applied to the turbulent channel flow. Its performance is compared with classical turbulence models as well as a neural network model that does not preserve Galilean invariance. A sensitivity study on the TBNN reveals that the network attempts to adjust to the dataset, but is limited by the mathematical form that guarantees Galilean invariance. △ Less

Submitted 5 December, 2018; originally announced December 2018.

Can phoretic particles swim in two dimensions?

Authors: David Sondak, Cory Hawley, Siyu Heng, Rebecca Vinsonhaler, Eric Lauga, Jean-Luc Thiffeault

Abstract: Artificial phoretic particles swim using self-generated gradients in chemical species (self-diffusiophoresis) or charges and currents (self-electrophoresis). These particles can be used to study the physics of collective motion in active matter and might have promising applications in bioengineering. In the case of self-diffusiophoresis, the classical physical model relies on a steady solution of… ▽ More Artificial phoretic particles swim using self-generated gradients in chemical species (self-diffusiophoresis) or charges and currents (self-electrophoresis). These particles can be used to study the physics of collective motion in active matter and might have promising applications in bioengineering. In the case of self-diffusiophoresis, the classical physical model relies on a steady solution of the diffusion equation, from which chemical gradients, phoretic flows and ultimately the swimming velocity, may be derived. Motivated by disk-shaped particles in thin films and under confinement, we examine the extension to two dimensions. Because the two-dimensional diffusion equation lacks a steady state with the correct boundary conditions, Laplace transforms must be used to study the long-time behavior of the problem and determine the swimming velocity. For fixed chemical fluxes on the particle surface, we find that the swimming velocity ultimately always decays logarithmically in time. In the case of finite Peclet numbers, we solve the full advection-diffusion equation numerically and show that this decay can be avoided by the particle moving to regions of unconsumed reactant. Finite advection thus regularizes the two-dimensional phoretic problem. △ Less

Submitted 30 November, 2016; originally announced November 2016.

Comments: 16 pages, 7 figures, RevTeX 4-1 style

Journal ref: Physical Review E 94, 062606 (2016)

Optimal Heat Transport in Rayleigh-Bénard Convection

Authors: David Sondak, Leslie M. Smith, Fabian Waleffe

Abstract: Steady flows that optimize heat transport are obtained for two-dimensional Rayleigh-Bénard convection with no-slip horizontal walls for a variety of Prandtl numbers $Pr$ and Rayleigh number up to $Ra\sim 10^9$. Power law scalings of $Nu\sim Ra^γ$ are observed with $γ\approx 0.31$, where the Nusselt number $Nu$ is a non-dimensional measure of the vertical heat transport. Any dependence of the scali… ▽ More Steady flows that optimize heat transport are obtained for two-dimensional Rayleigh-Bénard convection with no-slip horizontal walls for a variety of Prandtl numbers $Pr$ and Rayleigh number up to $Ra\sim 10^9$. Power law scalings of $Nu\sim Ra^γ$ are observed with $γ\approx 0.31$, where the Nusselt number $Nu$ is a non-dimensional measure of the vertical heat transport. Any dependence of the scaling exponent on $Pr$ is found to be extremely weak. On the other hand, the presence of two local maxima of $Nu$ with different horizontal wavenumbers at the same $Ra$ leads to the emergence of two different flow structures as candidates for optimizing the heat transport. For $Pr \lesssim 7$, optimal transport is achieved at the smaller maximal wavenumber. In these fluids, the optimal structure is a plume of warm rising fluid which spawns left/right horizontal arms near the top of the channel, leading to downdrafts adjacent to the central updraft. For $Pr > 7$ at high-enough Ra, the optimal structure is a single updraft absent significant horizontal structure, and characterized by the larger maximal wavenumber. △ Less

Submitted 11 July, 2015; originally announced July 2015.

Comments: 29 pages, 16 figures, submitted to the Journal of Fluid Mechanics

MSC Class: 76R10; 76E06; 76F25

A new class of finite element variational multiscale turbulence models for incompressible magnetohydrodynamics

Authors: David Sondak, John N. Shadid, Assad A. Oberai, Roger P. Pawlowski, Eric C. Cyr, Tom M. Smith

Abstract: New large eddy simulation (LES) turbulence models for incompressible magnetohydrodynamics (MHD) derived from the variational multiscale (VMS) formulation for finite element simulations are introduced. The new models include the variational multiscale formulation, a residual-based eddy viscosity model, and a mixed model that combines both of these component models. Each model contains terms that ar… ▽ More New large eddy simulation (LES) turbulence models for incompressible magnetohydrodynamics (MHD) derived from the variational multiscale (VMS) formulation for finite element simulations are introduced. The new models include the variational multiscale formulation, a residual-based eddy viscosity model, and a mixed model that combines both of these component models. Each model contains terms that are proportional to the residual of the incompressible MHD equations and is therefore numerically consistent. Moreover, each model is also dynamic, in that its effect vanishes when this residual is small. The new models are tested on the decaying MHD Taylor Green vortex at low and high Reynolds numbers. The evaluation of the models is based on comparisons with available data from direct numerical simulations (DNS) of the time evolution of energies as well as energy spectra at various discrete times. A numerical study, on a sequence of meshes, is presented that demonstrates that the large eddy simulation approaches the DNS solution for these quantities with spatial mesh refinement. △ Less

Submitted 2 December, 2014; originally announced December 2014.

Comments: 30 pages, 14 figures, submitted to the Journal of Computational Physics

MSC Class: 76F65 (Primary) 76W05 (Secondary)