## Xiang Cheng : Transformers learn in-context by (functional) gradient descent

- Applied Math and Analysis ( 0 Views )Motivated by the in-context learning phenomenon, we investigate how the Transformer neural network can implement learning algorithms in its forward pass. We show that a linear Transformer naturally learns to implement gradient descent, which enables it to learn linear functions in-context. More generally, we show that a non-linear Transformer can implement functional gradient descent with respect to some RKHS metric, which allows it to learn a broad class of functions in-context. Additionally, we show that the RKHS metric is determined by the choice of attention activation, and that the optimal choice of attention activation depends in a natural way on the class of functions that need to be learned. I will end by discussing some implications of our results for the choice and design of Transformer architectures.

## Hongkai Zhao : Mathematical and numerical understanding of neural networks: from representation to learning dynamics

- Applied Math and Analysis ( 0 Views )In this talk I will present both mathematical and numerical analysis as well as experiments to study a few basic computational issues in using neural network to approximate functions: (1) the numerical error that can be achieved given a finite machine precision, (2) the learning dynamics and computation cost to achieve certain accuracy, and (3) structured and balanced approximation. These issues are investigated for both approximation and optimization in asymptotic and non-asymptotic regimes.

## Sanchit Chaturvedi : Phase mixing in astrophysical plasmas with an external Kepler potential

- Applied Math and Analysis ( 85 Views )In Newtonian gravity, a self-gravitating gas around a massive object such as a star or a planet is modeled via Vlasov Poisson equation with an external Kepler potential. The presence of this attractive potential allows for bounded trajectories along which the gas neither falls in towards the object or escape to infinity. We focus on this regime and prove first a linear phase mixing result in 3D outside symmetry with exact Kepler potential. Then we also prove a long-time nonlinear phase mixing result in spherical symmetry. The mechanism is phenomenologically similar to Landau damping on a torus but mathematically the situation is quite a lot more complex. This is based on an upcoming joint work with Jonathan Luk at Stanford.

## Vakhtang Poutkaradze : Lie-Poisson Neural Networks (LPNets): Data-Based Computing of Hamiltonian Systems with Symmetries

- Applied Math and Analysis ( 57 Views )Physics-Informed Neural Networks (PINNs) have received much attention recently due to their potential for high-performance computations for complex physical systems, including data-based computing, systems with unknown parameters, and others. The idea of PINNs is to approximate the equations and boundary and initial conditions through a loss function for a neural network. PINNs combine the efficiency of data-based prediction with the accuracy and insights provided by the physical models. However, applications of these methods to predict the long-term evolution of systems with little friction, such as many systems encountered in space exploration, oceanography/climate, and many other fields, need extra care as the errors tend to accumulate, and the results may quickly become unreliable. We provide a solution to the problem of data-based computation of Hamiltonian systems utilizing symmetry methods. Many Hamiltonian systems with symmetry can be written as a Lie-Poisson system, where the underlying symmetry defines the Poisson bracket. For data-based computing of such systems, we design the Lie-Poisson neural networks (LPNets). We consider the Poisson bracket structure primary and require it to be satisfied exactly, whereas the Hamiltonian, only known from physics, can be satisfied approximately. By design, the method preserves all special integrals of the bracket (Casimirs) to machine precision. LPNets yield an efficient and promising computational method for many particular cases, such as rigid body or satellite motion (the case of SO(3) group), Kirchhoff's equations for an underwater vehicle (SE(3) group), and others. Joint work with Chris Eldred (Sandia National Lab), Francois Gay-Balmaz (CNRS and ENS, France), and Sophia Huraka (U Alberta). The work was partially supported by an NSERC Discovery grant.

## Barbara Keyfitz : Regular Reflection of Weak Shocks

- Applied Math and Analysis ( 28 Views )In joint work, Suncica Canic, Eun Heui Kim and I have recently proved the existence of a local solution to the regular reflection problem in the unsteady transonic small disturbance (UTSD) model for shock reflection by a wedge. There are two kinds of regular reflection, weak and strong, which are distinguished by whether the state immediately behind the reflected shock is subsonic (strong) or supersonic and constant, becoming subsonic further downstream (weak). In the more complicated case of weak regular reflection, the equation, in self-similar coordinates, is degenerate at the sonic line. The reflected shock becomes transonic and begins to curve there; its position is the solution to a free boundary problem for the degenerate equation.

We combine techniques which have been developed for solving degenerate elliptic equations arising in self-similar reductions of hyperbolic conservation laws with an approach to solving free boundary problems of the type that arise from Rankine-Hugoniot relations. Although our construction is limited to a finite part of the unbounded subsonic region, it suggests that this approach has the potential to solve a variety of problems in weak shock reflection.

## David Cai : Spatiotemporal Chaos, Weak Turbulence, Solitonic Turbulence & Invariant Measures - Statistical Characterization of Nonlinear Waves

- Applied Math and Analysis ( 29 Views )We will present an overview of the program of statistical description of long time, large scale dynamics of nonlinear waves and highlight some results we obtained: from vanishing mutual information measures in the spatiotemporal chaos induced by hyperbolic structures of PDEs, to confirmation of weak-turbulence spectra, and role of coherent structures in controlling energy transfer in turbulent cycles described by multiple cascade spectra, to effective stochastic dynamics. We will address the issue of how to obtain invariant measures for these systems. Finally, we will report on our study of statistical properties of the focusing nonlinear SchrÃ¶dinger equation, in the limit of a large number of solitons, corresponding to the semi-classical limit in a periodic domain. Our results demonstrate that the dynamics is described solitonic turbulence and there is a power law for the energy spectrum in the regime. We will discuss the connection between the wave turbulence and solitonic turbulence.

## Karl Glasner : Dissipative fluid systems and gradient flows

- Applied Math and Analysis ( 33 Views )This talk describes the the gradient flow nature of dissipative fluid interface problems. Intuitively, the gradient of a functional is given by the direction of ``steepest descent''. This notion, however, depends on the geometry assigned to the underlying function space. The task is therefore to find a metric appropriate for the given dynamics.

For the problem of surface tension driven Hele-shaw flow, the correct metric turns out to have a remarkable connection to an optimal transport problem. This connection points the way to a diffuse interface description of Hele-Shaw flow, given by a degenerate Cahn-Hilliard equation. Some computational examples of this model will be given. The problem of viscous sintering, the Stokes flow counterpart to the Hele-Shaw problem, will also be discussed.

## Anna Georgieva : Resonances in Nonlinear Discrete Periodic Medium

- Applied Math and Analysis ( 28 Views )We derive traveling wave solutions in a nonlinear
diatomic particle chain near the 1:2 resonance (k_{*}, omega_{*}), where omega_{*}=D(k_{*}), 2omega_{*}=D(2k_{*})
and omega=D(k) is the linear dispersion relation.
To leading order, the waves have form +/- epsilon sin(k n-omega t) + delta sin(2 k n-2 omega t), where the near-resonant acoustic frequency
omega and the amplitude epsilon of the first harmonic
are given to first order in terms of the
wavenumber difference k-k_{*} and the amplitude delta of the second harmonic. These traveling wave solutions are unique within a certain set of symmetries.

We find that there is a continuous line in parameter space, that transfers energy from the first to the second harmonic, even in cases where initially almost all energy is in the first harmonic, connecting these waves to pure optical waves that have no first harmonic content. The analysis is extended to higher resonances.

## Jade Vinson : The Holyhedron Problem

- Applied Math and Analysis ( 32 Views )Several years ago John Conway asked whether there could exist a polyhedron that "had a hole in every face," and coined the name "holyhedron" for them, if they should exist. We answer this question by constructing a holyhedron with 78,585,627 faces and genus 60,380,421. This is a polyhedron so that the interior of every face is connected but not simply connected.

## Michael Shearer : Continuum Models of Granular Flow

- Applied Math and Analysis ( 29 Views )Continuum models of the flow of granular materials in a hopper admit so-called radial solutions. These describe steady flows that appear realistic, and have been used extensively to design commercial hoppers. However, numerical results demonstrate that these solutions may not be robust to perturbation. Moreover, the time dependent equations are (notoriously) ill-posed. In this talk, I describe preliminary research designed to investigate the extent to which steady solutions may be used to represent granular flow. Using a combination of analysis and numerical experiments, we have explored simple models that are linearly ill posed. While there may be a stable steady state, it is a solution of a discretized continuum model, rather than the original equations. Moreover, the survival time of transients is inversely related to the mesh width, suggesting that the continuum limit is meaningless. While these results are not intended to invalidate the radial solutions, they do raise serious concerns about continuum modeling, and the possibility of designing a robust code that can be used to simulate a variety of granular flows.

## Valery A. Kholodnyi : Foreign Exchange Option Symmetry and a Coordinate-Free Description of a Foreign Exchange Option Market

- Applied Math and Analysis ( 28 Views )In spite of the fact that symmetries play one of the major roles in physics, the ir usage in finance is relatively new and, to the best of our knowledge, can be traced to 1995 when Kholodnyi introduced the beliefs-preferences gauge symmetry. In this talk we present another symmetry, foreign exchange option symmetry, int roduced by Kholodnyi and Price in 1996. Foreign exchange option symmetry associa tes financially equivalent options on opposite sides of the foreign exchange mar ket. In a two-currency market, the foreign exchange option symmetry is formalized in terms of the one-dimensional Kelvin transform. In a multiple-currency market the foreign exchange option symmetry is formalized in terms of differential geometr y on graphs, that is, in terms of vector lattice bundles on graphs and connectio ns on these bundles. Foreign exchange option symmetry requires no assumptions on the nature of a prob ability distribution for exchange rates. In fact, it does not even require the a ssumptions of the existence of such a distribution. Furthermore, the symmetry is applicable not only to a foreign exchange market but to any financial market as well. The practical applications of the foreign exchange option symmetry range from th e detection of a new type of true arbitrage to the detection of inconsistent mod els of foreign exchange option markets and the development of algorithms and sof tware to value and analyze portfolios of foreign exchange options.

## E. Bruce Pitman : Tubuloglomerular Feedback-Mediated Dynamics in Two Coupled Nephrons

- Applied Math and Analysis ( 31 Views )Previously, we developed a ``minimal'' dynamic model for the tubuloglomerular feedback (TGF) system in a single, short-looped nephron of the mammalian kidney. In that model, a semilinear hyperbolic partial differential equation was used to represent two fundamental processes of mass transport in the nephron's thick ascending limb (TAL): chloride advection by fluid flow through the TAL lumen and transepithelial chloride transport from the lumen to the interstitium. An empirical function and a time delay were used to relate nephron glomerular filtration rate to the chloride concentration at the macula densa of the TAL. Analysis of the model equations indicated that limit-cycle oscillations (LCO) in nephron fluid flow and chloride concentration can emerge for suffficiently large feedback gain and time delay. In this study, the single-nephron model has been extended to two nephrons, which are coupled through their filtration rates. Explicit analytical conditions were obtained for bifurcation loci corresponding to two special cases: (1) identical time-delays, but differing gains, and (2) identical feedback gain magnitudes, but differing time delays. Similar to the case of a single nephron, the analysis indicates that LCO can emerge in coupled nephrons for sufficiently large gains and delays. However, these LCO may emerge at lower values of the feedback gain, relative to a single (i.e., uncoupled) nephron, or at shorter delays, provided the delays are sufficiently close. These results suggest that, in vivo, if two nephrons are sufficiently similar, then coupling will tend to increase the likelihood of LCO. (In collaboration with Roman M. Zaritski, Leon C. Moore and H. E. Layton)

## Stephen J. Watson : A Priori Bounds in Thermo-Viscoelasticity with Phase Transitions

- Applied Math and Analysis ( 26 Views )The object of our study is the set of equations of thermo-elasticity with viscosity and heat conduction. These equations include, as a special case, the compressible Navier-Stokes equation familiar from gas dynamics, but in addition allow for solid-like materials. We seek to understand the temporal asymptotic fate of large initial data under a variety of boundary conditions. The realm of phase changes, such as occur in Van-der-Waals gases and martensitic transformations, are of especial interest. Now, obtaining point-wise a priori bounds on the density which are time independent is a major analytical obstacle to resolving this question. We present two new results on this issue. First, for specified-stress boundary conditions we give a positive result applicable to a general class of materials. Second, for Dirichlet boundary conditions we derive the estimates for a special class of gaseous materials; p'th power gases. We conclude with a discussion on the relation between asymptotic states and minimization principles of associated free energies. Numerical simulations will highlight some surprising features of the dynamics. In particular, the limiting states are not necessarily strong minimizers, in the sense of the calculus of variations, of the free energy.

## Graham Wilks : Heated Jet Assimilation into External Streams

- Applied Math and Analysis ( 32 Views )The assimilation of a simultaneous momentum and heat source into aligned uniform and non-uniform streams is considered.The governing boundary layer equations are transformed utilising aggregate properties of the flow field with respect to the excesses of heat and momentum flux at the source relative to the external stream.A non-dimensional downstream coordinate Ã? reflecting the local relative importance of jet to external stream characteristics provides a unified framework within which to investigate the entire semi-infinite flow field downstream of the source.The problems examined devolve down to transitional flow in Ã? between acknowledged strong jet and weak jet regimes in the immediate vicinity of and downstream of the source respectively.Perturbation solutions are developed in the two regimes.The downstream asymptotic velocity and temperature profiles are shown to be associated with new solutions of the Falkner-Skan equation subject to the boundary condition of symmetry,as opposed to no slip, at the axis of symmetry.A stability analysis of the new solutions and comprehensive numerical solutions over the full flow field confirm that there may be physically realisable flows in which a residual jet identity remains distinguishable within the downstream flow.

## Edward Belbruno : Low Energy Trajectories in Celestial Mechanics and Stability Transition Regions With Applications to Astronomy and Space Travel

- Applied Math and Analysis ( 32 Views )In the past two decades a new type of chaotic dynamics has been noticed in the three and four body problems which has not been understood. In 1986, using a numerical algorithm, an interesting region supporting chaotic motion was discovered about the moon, under the perturbation of the earth. This region is now termed the weak stability boundary. New types of dynamics were subsequently discovered near this boundary. These dynamics have the property that they give rise to very low energy trajectories with many important applications. In 1991, a new type of low energy trajectory to the moon was discovered which was used to place a Japanese spacecraft, Hiten, in orbit about the moon in October of that year. This was the first application of this type of dynamics to space travel. These low energy trajectories, so called WSB transfers, are now being planned by NASA, Europe and Japan for several new missions to the moon, Europa, Mars. Motion near this boundary also gives rise to an interesting resonance transition dynamics, and work by the speaker with Brian Marsden at Harvard is discussed in its relevance to short period comets, and Kuiper belt objects. An analytic representation for this boundary is also presented and its connections with heteroclinic intersections of hyperbolic invariant manifolds is discussed. If there is time, a new type of periodic motion for Hill's problem is looked at.

## Thomas Y. Hou : Singularity Formation in 3-D Vortex Sheets

- Applied Math and Analysis ( 30 Views )One of the classical examples of hydrodynamic instability occurs when two fluids are separated by a free surface across which the tangential velocity has a jump discontinuity. This is called Kelvin-Helmholtz Instability. Kelvin-Helmholtz instability is a fundamental instability of incompressible fluid flow at high Reynolds number. The idealization of a shear layered flow as a vortex sheet separating two regions of potential flow has often been used as a model to study mixing properties, boundary layers and coherent structures of fluids. In a joint work with G. Hu and P. Zhang, we study the singularity of 3-D vortex sheets using a new approach. First, we derive a leading order approximation to the boundary integral equation governing the 3-D vortex sheet. This leading order equation captures the most singular contribution of the integral equation. Moreover, after applying a transformation to the physical variables, we found that this leading order 3-D vortex sheet equation de-generates into a two-dimensional vortex sheet equation in the direction of the tangential velocity jump. This rather surprising result confirms that the tangential velocity jump is the physical driving force of the vortex sheet singularities. It also shows that the singularity type of the three-dimensional problem is similar to that of the two-dimensional problem. Detailed numerical study will be provided to support the analytical results, and to reveal the generic form and the three-dimensional nature of the vortex sheet singularity.

## B. Scott Gaudi : Microlensing and the Search for Extrasolar Planets

- Applied Math and Analysis ( 29 Views )The PLANET collaboration has monitored nearly 100 microlensing events of which more than 20 have the sensitivity required to detect perturbations due to a planetary companion to the primary lens. No planets have been detected. These null results indicate that Jupiter mass planets with separations from 1-5 AU are not common -- the first such limits for extrasolar planets at these separations by any technique. While interpretation of null results is not trivial, interpretation of future detections will be substantially more difficult, due to degeneracies among the planetary fit parameters and degeneracies with perturbations due to other, non-planetary phenomena. The analysis is further complicated by the unusual situation that observational strategies are altered real-time when perturbations are detected. I discuss these difficulties and present methods to cope with them. Finally, I discuss future prospects for microlensing planet searches.

## John Lavery : L_1 Splines: Shape-Preserving, Multiscale, Piecewise Polynomial Geometric Modeling

- Applied Math and Analysis ( 29 Views )We discuss a new class of cubic interpolating and approximating
"L_{1} splines" that preserve the shape both of smooth data and
of data with
abrupt changes in magnitude or spacing. The coefficients of these splines
are calculated by minimizing the L_{1}
norm of the second derivatives. These
splines do not require constraints, penalties, a posteriori filtering or
interaction with the user. Univariate and multivariate cases are treated in
one and the same framework. L_{1} splines are implemented using efficient
interior-point methods for linear programs.

## Oscar Gonzalez : Global Curvature, Ideal Knots and Models of DNA Self-Contact

- Applied Math and Analysis ( 29 Views )Experiments on knotted DNA molecules suggest that certain physical properties of DNA knots can be predicted from a corresponding ideal shape. Intuitively, when a given knot in a piece of string is pulled tight, it always achieves roughly the same geometrical configuration, with a minimum length of string within the knot. Such a configuration is called an ideal shape for the knot, and approximations of ideal shapes in this sense have been found via a series of computer experiments. These shapes have intriguing physical features and have been shown to capture average properties of knotted polymers. But when does a shape satisfy the intuitive geometrical definition for ideality? In this talk I show that ideal shapes can be understood using only elementary (but new!) mathematics. In particular, I show that global curvature, a very natural and simple generalization of the classic concept of local curvature, leads to a simple characterization of an ideal shape and to a necessary condition for ideality. Another application of global curvature can be found in characterizing the equilibria of knotted curves or rods, which may exhibit self-contact after sufficient twisting. Here global curvature provides a simple way to formulate the constraint that prevents a rod from passing through itself.

## Juan M. Restrepo : Wave-driven Ocean Circulation

- Applied Math and Analysis ( 25 Views )After the sun, the oceans are the most significant contributor to our climate. Oceanic surface gravity waves are thought to have no influence on the global circulation of the oceans. However, oceanic surface gravity waves have a mean Lagrangian motion, the Stokes drift. This talk will present preliminary results that suggest that the dynamics of basin-scale oceanic currents are modified by the presence of the Stokes drift. In places where the Stokes drift is significant, it is possible that the ocean circulation, and hence climate, is not entirely well captured by present day models of the general circulation.

Special day: Thursday, 4pm, Room 120 Physics

## Tasso J. Kaper : Strong Pulse Interactions in Coupled Reaction-Diffusion Systems

- Applied Math and Analysis ( 25 Views )A plethora of biological and chemical pattern formation problems are modeled using coupled reaction-diffusion equations of activator-inhibitor type. In 1993, the new phenomenon of self- -replicating spots and pulses was discovered in the Gray-Scott model and in a ferrocyanide reaction it models, by John Pearson and Harry Swinney and collaborators. In this talk, we present an analysis of pulse splitting. Furthermore, it turns out that the perturbation theory developed for the Gray-Scott analysis can be extended in a natural way to analyze a general class of coupled activator-inhibitor systems, including the Gierer-Meinhart and Schnakenberg models, in order to determine whether pulses attract or repel each other, and if they repel, whether they also the undergo self-replication. The work may be classified as a treatment of the moderately strong and strong pulse interaction problem. This work is part of a larger collaborative project with Arjen Doelman, Wiktor Eckhaus, Rob Gardner and my student Dave Morgan. We will conclude with some open questions.

## Donald J. Estep : Estimating the Error of Numerical Solutions of Systems of Reaction-Diffusion Equations

- Applied Math and Analysis ( 29 Views )One of the pressing problems in the analysis of reaction-diffusion equations is obtaining accurate and reliable estimates of the error of numerical solutions. Recently, we made significant progress using a new approach that at the heart is computational rather than analytical. I will describe a framework for deriving and analyzing a posteriori error estimates, discuss practical details of the implementation of the theory, and illustrate the error estimation using a variety of well-known models. I will also briefly describe an application of the theory to the class of problems that admit invariant rectangles and discuss the preservation of invariant rectangles under discretization.

## Peter K. Moore : An Adaptive H-Refinement Finite Element For Solving Systems of Parabolic Partial Differential Equations in Three Space Dimensions

- Applied Math and Analysis ( 29 Views )Adaptive methods for solving systems of partial differential equations have become widespread. Robust adaptive software for solving parabolic systems in one and two space dimensions is now widely available. Three spatial adaptive strategies and combinations thereof are frequently employed: mesh refinement (h-refinement); mesh motion (r-refinement); and order variation (p-refinement). These adaptive strategies are driven by a priori and a posteriori error estimates. I will present an adaptive h-refinement finite element code in three dimensions on structured grids. These structured grids contain irregular nodes. Solution values at these nodes are determined by continuity requirements across element boundaries rather than by the differential equations. The differential-algebraic system resulting from the spatial discretization is integrated using Linda Petzold's multistep DAE code DASPK. The large linear systems resulting from Newton's method applied to nonlinear system of differential algebraic equations is solved using preconditioned GMRES. In DASPK the matrix-vector products needed by GMRES are approximated by a ``directional derivative''. Thus, the Jacobian matrix need not be assembled. However, this approach is inefficient. I have modified DASPK to compute the matrix-vector product using stored Jacobian matrix. As in the earlier version of DASPK, DASSL, this matrix is kept for several time steps before being updated. I will discuss appropriate preconditioning strategies, including fast-banded preconditioners. In three dimensions when using multistep methods for time integration it is crucial to use a ``warm restart'', that is, to restart the dae solver at the current time step and order. This requires interpolation of the history information. The interpolation must be done in such a way that mode irregularity is enforced on the new grid. A posteriori error estimates on uniform grids can easily be generalized from two-dimensional results (Babuska and Yu showed that in the case of odd order elements, jumps across elemental boundaries give accurate estimates, and in the case of even order elements, local parabolic systems must be solved to obtain accurate estimates). Babuska's work can even be generalized to meshes with irregular modes but now they no longer converge to the true error (in the case of odd order elements). I have developed a new set of estimates that extend the work of Babuska to irregular meshes and finite difference methods. These estimates provide a posteriori error indicators in the finite element context. Several examples that demonstrate the effectiveness of the code will be given.

## Kirill B. Skouibine : The Role of Cardiac Tissue Structure in Reentry Induction: A Modeling Study

- Applied Math and Analysis ( 27 Views )Most dangerous cardiac arrhythmia, ventricular fibrillation (VF), is characterized by chaotic electrical behavior of the tissue. At the onset of the first, more organized stage of VF waves of electric activity in the heart become reentrant leading to fast irregular contraction. Better understanding of the mechanisms underlying early VF events will lead to more efficient treatment. Reentry induction has been performed in several experiments. We devise a model of cardiac tissue and use it to obtain a close match to the experimental results. The model combines macroscopic and microscopic properties of cardiac tissue.