Course description
This course introduces methods that range from techniques for systems of linear equations, nonlinear equations, approximation of functions, interpolation, clustering, least square data fitting and classification, differentiation and integration. More emphasis is put on applied linear algebra topics which are prerequisite for Artificial Intelligence, Machine Learning, and other advanced courses. We make use of NumPy programming to implement and analyze the methods.
Course content
a. Approximation Errors and Approximating Single Variable Functions
b. Finding Roots, Numerical Integration, Differentiation, Optimization of Single Variable Functions
c. Direct Methods for Solving System of Linear Equations
d. Interpolation
e. Data Representation - Vectors and Matrices
f. Approximation of Multivariable Functions
g. Norms, Distance, KNN classification, K-Means Clustering and Simple Linear Regression
h. Orthogonality and Least Square Methods
Learning outcomes
- CLO1: Demonstrate understanding of the Linear Algebra and Calculus theory that underlies many of the common computational methods and how the methods are used to obtain approximate solutions to otherwise intractable mathematical problems.
- CLO2: Apply numerical methods to obtain approximate solutions to mathematical problems. Derive and analyze numerical methods for various mathematical operations and tasks, such as interpolation, differentiation, integration, the solution of linear and nonlinear equations, and least square optimization used in clustering, data fitting, and classification.
- CLO3: Implement numerical methods in NumPy. Write efficient, well-documented NumPy code and present numerical results in an informative way.
Topics
Week | Topic | Teaching-Learning Strategy | Assessment Strategy | Corresponding CLOs |
|---|---|---|---|---|
1 | Review of calculus: derivatives, Taylor polynomial approximation, finding optima of single variable functions, Basics of NumPy | Lecture (3h) Lab (1.5h) | Class Test Midterm Exam Lab work | CLO1, 3 |
2-3 | Approximating derivatives – forward, backward, and central difference, Finding roots of single variable functions – Bisection, False position, Secant and Newton-Raphson Method | Lecture (3h) Lab (1.5h) | Class Test Midterm Exam Lab work | CLO2, 3 |
4-5 | Finding optima – Gradient Descent and Newton’s method, Approximating integration – Trapezoidal rule, Simpson’s rule and Lagrange Interpolation | Lecture (3h) Lab (1.5h) | Class Test Midterm Exam Lab work | CLO2, 3 |
6 | Review of Gaussian elimination, and LU decomposition, Inverses, Applications: Polynomial interpolation and Vandermonde matrix, applications of solving system of linear equations | Lecture (3h) Lab (1.5h) | Class Test Midterm Exam Lab work | CLO1 |
7 | Vectors - review of vector notation, vector operations, linear and affine multivariable functions, complexity of vector computations | Lecture (3h) Lab (1.5h) | Class Test Final Exam Lab work | CLO1 |
7 | Applications: vector representation of data (e.g., images, documents, timeseries, features), vector representation of linear and affine functions (e.g., regression, Linear (Taylor) approximation of multivariable functions) | Lecture (3h) Lab (1.5h) | Class Test Final Exam Lab work | CLO1, 3 |
8 | Norms and distances - Euclidean norm and distances, properties (Cauchy-Schwarz and triangle inequalities, Pythagorean theorem), Statistical measurements of data: average, rms, standard deviation, and angle between vectors and correlation, covariance; representation of hyperplanes, Partial Derivatives, Directional Derivatives, Gradient Descent/Ascent. | Lecture (3h) Lab (1.5h) | Class Test Final Exam Lab work | CLO2, 3 |
9 | Applications: K-nearest neighbor classification, single variable linear regression, k-means clustering | Lecture (3h) Lab (1.5h) | Lab work | CLO2, 3 |
10 | Basis, orthogonality and inner products: basis and change of basis, Orthogonal basis, Gram-Schmidt, modified Gram-Schmidt algorithms, QR decomposition of matrices | Lecture (3h) Lab (1.5h) | Class Test Final Exam Assignment | CLO1 |
11 | Linear least-squares: solution to overdetermined systems, normal equation and pseudo inverse of a matrix, Computing pseudo inverse using QR and Cholesky factorization, solving least squares using matrix-vector derivatives | Lecture (3h) Lab (1.5h) | Class Test Final Exam Lab work | CLO2, 3 |
11 | Data fitting and least-square regression, feature engineering, Least-square classification, regularized least square data fitting, least square function approximation | Lecture (3h) Lab (1.5h) | Class Test Final Exam Lab work | CLO3 |
12 | Problem Condition, Algorithm Stability | Lecture (3h) | Assignment | CLO2 |
13 | Review | Lecture (3h) Lab (2h) |
Textbooks
- [VMLS] Introduction to Applied Linear Algebra – Vectors, Matrices, and Least Squares, by S. Boyd and L. Vandenberghe. Available at https://web.stanford.edu/~boyd/vmls/
- [NME] Numerical Methods for Engineers, by S. Chapra, 7th ed., McGraw Hill
- [LAB] Python Programming and Numerical Methods: A Guide for Engineers and Scientists, by Qingkai Kong, Timmy Siauw and Alexandre M. Bayen, Academic Press
References
- Applied Numerical Methods with MATLAB for Engineers and Scientists, by S. Chapra, 3rd ed., McGraw Hill.
- [notes] Additional notes to Applied Numerical Computing, http://www.seas.ucla.edu/~vandenbe/133A/133A-notes.pdf
- Linear Algebra and Its Applications, D. Lay et al.
- Calculus: Early Transcendentals, James Stewart.
Similar course
- Applied Numerical Computing, Prof. L. Vandenberghe, UCLA