Numerical Analysis from the Bottom-Up

I have attended and sat in on many numerical analysis and numerical methods courses. In fact, a pair of beautiful courses taught by Professor Krzysztof Sikorski (, was responsible for my own research into numerical methods for PDEs.

That said, second courses in numerical methods (numerics for ODEs and PDEs) always seem to be taught in a scattered fashion: there are a large number of topics to cover in numerical ODEs and PDEs, and these courses valiantly attempt to go over them all. However, in my experience, these courses have only ever commented on the unifying principles underlying all these numerical methods. As a result, one's numerics education is a scattered experience, and we are forced to put together the pieces ourselves afterwards when doing research.

My goal this semester (Spring 2016) was to do this differently. While teaching Numerical Analysis II, I decided to build the course bottom-up. The goal was to identify a unifying principle (or principles) to all the numerical methods for ODEs and PDEs. I realized, partly due to my own research experience, that the most sensible explicit unifying principle is polynomial interpolation.

Starting with polynomial interpolation, it is possible to derive finite differences, and quadrature rules. Using those as building blocks, one can derive all the multistep and multistage methods for ODEs. It is also quite easy to derive error estimates for these methods using the polynomial interpolation error formula, an approach that seems surprisingly rare (though some sources do take this approach).

For numerical PDEs, it is straightforward to start with the Method of Weighted Residuals, and derive all collocation and Galerkin methods from there. Further, by picking a polynomial/piecewise polynomial basis (regardless of the collocation/Galerkin framework), it becomes straightforward to derive spectral methods, finite difference methods and finite element methods.

I make no claims that this is the best way to teach numerical analysis. However, I think it is worth experimenting with this bottom-up approach. To that end, I've been writing my own texts in Latex and posting them up on a class webpage. Hopefully, over time, these documents will evolve into something more polished and rigorous.