Tricks and Tips in Numerical Computing - Keynote

Dive into a comprehensive 56-minute keynote from JuliaCon 2018 featuring Nick Higham, Royal Society Research Professor and Richardson Professor of Applied Mathematics at the University of Manchester. Explore a wealth of tricks and tips in numerical computing, covering topics such as differentiation techniques, complex plane computations, matrix operations, and low precision arithmetic. Learn about the complex step method, unwinding numbers, roundtrip relations, and the matrix chain multiplication problem. Gain insights into IEEE Standard 754, rounding error analysis, and the applications of half-precision arithmetic. The talk concludes with a Q&A session addressing randomization, method selection, and the relationship between low precision and power consumption.

Welcome.
Introducing the speaker.
What are tricks and tips?.
Differentiation with(out) a difference.
V-shape curve is a result of floating-point evaluation (cancelation) errors dominating truncation errors.
"Automatic differentiation ".
Complex step method.
Example: derivative of atan(x)/(1 + e^(-x^2)) at x = 2.
Computing principal logarithm in a complex plane, a multi-valued function.
Computing the principle logarithm in the 1960s.
Logarithm of the product of numbers, complex case.
Arcsin and Arccos in complex plane.
Unwinding number.
Roundtrip relations.
Accurate difference.
Low rank updated of n x n real matrix A.
Why Sherman-Morrison formula holds?.
World's Most Fundamental Matrix Equation.
Computing a product.
Matrix chain multiplication problem (MCMP).
Chain rule of differentiation and MCMP.
Randomization.
1985 IEEE Standard 754 and it 2008 Revision.
Model for rounding errors analysis.
This model is weaker than what IEEE Standard actually says.
Model vs correctly rounded result.
Prevision versus accuracy.
Accuracy is not limited by the precision.
Photocopying errors.
Typing errors.
Low precision arithmetic.
Applications of half-precision (fp16, floating point 16 bits).
Error analysis in low precision arithmetic.
What you can do to reduce error in fp16?.
Can we obtain more information bounds?.
Conclusions.
Q&A: how to avoid the case when randomization makes the problem worse?.
Q&A: how to choose between methods like contour integral and higher precision arithmetic?.
Q&A: does half-precision allow a brute force analysis of the distribution of operations?.
Q&A: can you comment on low precision and power consumption?.