Jacobi’s, Gauss-Seidel, and Relaxation Methods

Definition

Jacobi’s, Gauss-Seidel, and Relaxation methods are iterative numerical techniques used to solve systems of linear algebraic equations of the form $Ax = b$. Unlike direct methods (like Gaussian elimination) that provide an exact solution in a finite number of steps, these methods start with an initial guess and refine it through successive approximations until the solution converges to a desired level of accuracy.

Main Content

1. Jacobi’s Method (Method of Simultaneous Displacement)

It is the simplest iterative method where the values of the variables obtained in the current iteration are not used until the next iteration is fully calculated.
It requires the system to be strictly diagonally dominant for convergence, meaning the absolute value of the diagonal element in each row must be greater than the sum of the absolute values of other elements in that row.

2. Gauss-Seidel Method (Method of Successive Displacement)

This is an improvement over the Jacobi method. It uses the most recently updated values as soon as they are available within the same iteration.
Because it uses updated values immediately, it generally converges faster than the Jacobi method and requires less computer memory.

3. Relaxation Method (Successive Over-Relaxation - SOR)

This method introduces an acceleration factor called the "relaxation parameter" ($\omega$) to speed up the convergence of the Gauss-Seidel method.
If $0 < \omega < 1$, it is called under-relaxation; if $1 < \omega < 2$, it is called over-relaxation. If $\omega = 1$, it reverts to the standard Gauss-Seidel method.

Iterative Convergence Flow:
[Initial Guess] -> [Calculation] -> [Check Error] -> [If > tol, repeat]
      ^                                     |
      |_____________________________________|

Working / Process

1. Initialization and Arrangement

Rearrange the system of equations so that the coefficient matrix is diagonally dominant.
Choose an initial guess for the variables, typically $x_1^{(0)} = x_2^{(0)} = ... = x_n^{(0)} = 0$.

2. Iterative Calculation

For Jacobi: Compute $x_i^{(k+1)}$ using only values from the previous iteration $x^{(k)}$.
For Gauss-Seidel: Compute $x_i^{(k+1)}$ using the latest values $(x_1^{(k+1)}, ..., x_{i-1}^{(k+1)}, x_{i+1}^{(k)}, ..., x_n^{(k)})$.
For Relaxation: Apply the formula $x_i^{(new)} = (1-\omega)x_i^{(old)} + \omega x_i^{(Gauss-Seidel)}$.

3. Convergence Check

Calculate the absolute difference between the new value and the previous value for each variable.
Stop the process when the difference is less than the predefined tolerance level ($\epsilon$).

Advantages / Applications

These methods are highly efficient for solving large, sparse systems of linear equations (where most coefficients are zero).
They are less prone to round-off error accumulation compared to direct methods in very large systems.
Widely used in engineering simulations, fluid dynamics, heat transfer analysis, and structural mechanics where massive matrices are common.

Summary

Jacobi's method updates all variables simultaneously using old values, while Gauss-Seidel uses the newest values immediately to speed up convergence.
The Relaxation method (SOR) applies a weight factor to the Gauss-Seidel result to force faster convergence or improve stability in difficult problems.
These iterative numerical methods provide an approximate solution through successive refinement until a specified tolerance is reached.
Important terms: Diagonal Dominance, Tolerance ($\epsilon$), Relaxation Parameter ($\omega$), and Convergence.