Measuring and Mitigating Forgetting when Extending a PDE Foundation Model to Simulated Flow Regimes

Abstract

Foundation models for partial differential equations adapt efficiently to new tasks, but their effect on the original pretraining distribution is rarely measured. We study this retention problem on Poseidon, a neural operator pretrained on periodic-domain compressible Euler and incompressible Navier–Stokes families. We adapt it to two regimes absent from pretraining, a compressible Kelvin–Helmholtz flow and wall-bounded flow past a circular cylinder. Fine-tuning adapts efficiently but degrades the pretraining families. LoRA still forgets substantially, and freezing the whole trunk reduces forgetting without preventing it. We show that low-weight replay and joint training resolve this trade-off, extending the model to the new regimes while keeping the pretraining families near their original accuracy. These results suggest that lightweight replay can extend a PDE foundation model while substantially preserving its existing capabilities.

Cite

@inproceedings{masood2026forgetting,
  title = {Measuring and Mitigating Forgetting when Extending a {PDE} Foundation Model to Simulated Flow Regimes},
  author = {Masood, Wasif and Noor, Shoaib},
  booktitle = {NeurIPS 2026 Workshop on Representations for the Physical Sciences},
  year = {2026},
  url = {https://empirischtech.at/papers/pde-forgetting}
}