Get hands On with PINNs

Main ideas / concepts conveyed

What PINNs are (recap)

A Physics-Informed Neural Network (PINN) learns a solution to a PDE by training a neural network using:

• Inputs: the spatial domain and time domain
• Loss components:
  • Residual loss from enforcing the PDE
  • Constraint loss from enforcing boundary/initial conditions

The neural network output is interpreted as the physical field (e.g., temperature in the heat equation). The PDE is enforced through derivatives of the network output with respect to the inputs.

---

Workflow for implementing a PINN (DeepXDE-based)

The lecture describes a step-by-step method to implement and train a PINN code by defining:

1. Computational geometry (space domain)
2. The PDE (expressed in a TensorFlow/DeepXDE-compatible form)
3. Boundary and initial conditions
4. Training/testing data sampling
5. Neural network architecture
6. Optimizer, initializer, compilation
7. Training and prediction

---

How to implement with DeepXDE (detailed methodology / instruction flow)

The presenter focuses on using DeepXDE, a Python library based on TensorFlow.

Installation options mentioned include:

• pip install deepxde (spelled oddly in subtitles as “dxd/deep sd”)
• conda installation
• GitHub installation

Dependency/framework context:

• DeepXDE is based on TensorFlow, though subtitles also mention “Pytorch and Tensorflow” (not fully consistent).

---

Step-by-step coding methodology (as described)

1) Define the computational domain

1D case (heat equation example)

• Spatial domain: interval ([-1, 1])
• Time domain: ([0, 1])
• Combined into a spatiotemporal domain (referred to as “time geometry” / “dual time”)

2D case (later example)

• Spatial domain: 2D rectangle with corners ((-1,-1)) to ((1,1))
• Time domain: ([0,1])
• Combined into a 3D input domain: ((x, y, t))

---

2) Define the PDE

The PDE must be expressed using derivatives of the neural network output with respect to inputs.

To compute derivatives, the lecture uses:

• Jacobian/Hessian utilities from DeepXDE (TensorFlow-readable)
• Later, tf.gradients to generalize (e.g., beyond 1D)

Example: 1D Heat equation

[ \frac{\partial u}{\partial t} = \alpha \frac{\partial^2 u}{\partial x^2} ]

Example: 1D Burgers’ equation

Presented in a rearranged form, including diffusion and nonlinear advection, e.g.:

• (\nu \frac{\partial^2 u}{\partial x^2} = \frac{\partial u}{\partial t} + u\frac{\partial u}{\partial x})

The implementation idea is to put terms on the left/right consistently.

---

3) Define boundary and initial conditions

• Boundary conditions: use DeepXDE boundary-condition functions (subtitles mention dde.bc)
• Initial conditions: use DeepXDE initial-condition functions (subtitles mention dde.ic)

These condition functions use:

• the spatiotemporal domain sampling, and
• the PINN output.

---

4) Assemble training data

Training/validation datasets include:

• PDE interior points (residual points)
• Boundary points
• Initial condition points

Sampling counts mentioned in subtitles:

Heat 1D example

• num_domain = 4000 (interior/residual)
• boundary points: 2000
• initial condition points: 1000
• total training points: 7000
• validation/monitoring: num_test = 1000 (random sampling for monitoring)

Burgers 1D example

• Count details are not fully clear, but the structure is similar.

2D heat example

• interior/residual points: 8000 (from “dual time”)
• boundary points: 2000
• initial points: 3000
• validation: 1000

---

5) Design neural network architecture

• Input size equals the number of independent variables:
  • 1D: ((x,t)) → input dimension = 2
  • 2D: ((x,y,t)) → input dimension = 3
• Output dimension equals the PDE unknown:
  • heat temperature (u) → output dimension = 1

Example architecture setting mentioned:

• 1D heat example: layer_size = 2 + 32*3 + 1
  • interpreted as 3 hidden dense layers with 32 nodes each

Activation function:

• tanh (described in subtitles as “t10 function” / “tanh function”), chosen because it is differentiable.

---

6) Initializer and optimizer

• Initializer: “glorot uniform” / “glory uniform” (Glorot/Xavier uniform)
• Optimizer: Adam
• Compilation: compile with:
  • model optimizer and learning rate
  • metric monitoring: L2 relative error (“L2 relative arrow” in subtitles)

---

7) Train the model

• Training produces a train_state / loss history.
• Validation/test metric is monitored.
• Iterations noted: epochs = 10,000 in the heat 1D example.

---

8) Predict solution and evaluate residual

1. Create a grid of test inputs ((x,t)) or ((x,y,t))
2. Compute predictions using:
   • model.predict(test_domain) → predicted (u)
3. Compute PDE residual using the PDE/residual computation operator (subtitles mention residual evaluation like op/pde residual)

Residual interpretation:

• residuals are described as close to zero, indicating PDE satisfaction.

---

Adaptive refinement concept (RAR)

The lecture briefly introduces RAR (Residual-based Adaptive Refinement):

• After training for some epochs, identify points with the largest residual (“worst” points)
• Add/replace those points in training data
• Retrain to improve accuracy

Reported outcome:

• residual stays around ~1%, described as ~99% accuracy

---

Key example equations and how they relate to implementation

Heat equation (1D)

• PDE: [ \frac{\partial u}{\partial t} = \alpha \frac{\partial^2 u}{\partial x^2} ]

• Geometry: 1D interval

• Network input: ((x,t))

---

Burgers equation (1D)

Includes:

• diffusion term (second derivative in (x))
• advection term (u\,\partial u/\partial x)

The PDE is formed by matching left/right sides in code.

---

Heat equation (2D)

PDE shown in subtitles (with stated coefficients): [ \frac{\partial u}{\partial t} = 0.5\left(\frac{\partial^2 u}{\partial x^2}\right)

+ \left(\frac{\partial^2 u}{\partial y^2}\right) ]

Derivative generalization:

• use gradients/Jacobians/Hessians to compute second derivatives w.r.t. x and y

Network input becomes ((x,y,t)):

• geometry is a 2D rectangle combined with time.

---

Main lesson about the 1D → 2D transition

To move from 1D PDEs to 2D PDEs in PINNs:

• Expand inputs: from ((x,t)) to ((x,y,t))
• Change geometry: from a 1D interval to a 2D rectangle
• Update network input dimension: from 2 to 3
• Compute additional derivatives: obtain both (\partial^2 u/\partial x^2) and (\partial^2 u/\partial y^2) using gradients/Jacobians/Hessians

---

Speakers / sources featured

• Jushen Yang (speaker; also named in subtitles as “my name is jushen yang”)
• Aditya Balu (speaker; subtitles show “my co-worker aditya b…” / “aditya bulu”)
• Iowa State University (affiliation mentioned; not necessarily a speaker)
• DeepXDE (software/library mentioned)
• TensorFlow (framework mentioned; via DeepXDE and tf.gradients)