Gaussian Elimination

Definition

The fundamental idea of Gaussian elimination is to add multiples of one equation to the others in order to eliminate a variable and to continue this process until only one variable is left. Once this final variable is determined, its value is substituted back into the other equations in order to evaluate the remaining unknowns. This method is a step‐by‐step elimination of the variables.

Who

Hey, it’s 2021 and these videos from 13 years ago (The Khan Academy) are saving my university economics degree, thank you

My math teacher totally confused the class with this principle.

My professor makes this seem so complicated but when you (the The Organic Chemistry Tutor) didn’t [make it complicated] it was so easy.

This guy (the The Organic Chemistry Tutor) explains it so well…my textbook explains it like a brick.

What

Gaussian Elimination helps to put a matrix in row echelon form, while Gauss-Jordan Elimination puts a matrix in reduced row echelon form. For small systems (or by hand), it is usually more convenient to use Gauss-Jordan elimination and explicitly solve for each variable represented in the matrix system. However, Gaussian elimination in itself is occasionally computationally more efficient for computers. Also, Gaussian elimination is all you need to determine the rank of a matrix (an important property of each matrix) while going through the trouble to put a matrix in reduced row echelon form is not worth it to only solve for the matrix’s rank.

EDIT: Here are some abbreviations to start off with: REF = “Row Echelon Form”. RREF = “Reduced Row Echelon Form.”

In your question, you say you reduce a matrix A to a diagonal matrix where every nonzero value equals 1. For this to happen, you must perform row operations to “pivot” along each entry along the diagonal. Such row operations usually involve multiplying/dividing by nonzero scalar multiples of the row, or adding/subtracting nonzero scalar multiples of one row from another row. My interpretation of REF is just doing row operations in such a way to avoid dividing rows by their pivot values (to make the pivot become 1). If you go through each pivot (the numbers along the diagonal) and divide those rows by their leading coefficient, then you will end up in RREF. See these Khan Academy videos for worked examples.

In a system Ax=B, x can only be solved for if A is invertible. Invertible matrices have several important properties. The most useful property for your question is that their RREF is the identity matrix (a matrix with only 1’s down the diagonal and 0’s everywhere else). If you row-reduce a matrix and it does not become an identity matrix in RREF, then that matrix was non-invertible. Non-invertible matrices (also known as singular matrices) are not as helpful when trying to solve a system exactly.¹

---

Gauss-Jordan elimination means you find the matrix inverse A⁻¹.

Gaussian elimination means you only find the solution to Ax=b.

When you have the matrix inverse, of course you can also find the solution x=A−1b, but this is more work.³

Gauss Elimination

Gauss Elimination Method is one of the most widely used methods. This method is a systematic process of eliminating unknowns from the linear equations.²

Gauss-Jordan Elimination

Gauss Jordan Method is a little modification of the Gauss Elimination Method. Here, during the stages of elimination, the coefficients are eliminated in such a way that the systems of equations are reduced to a diagonal matrix. The very first method of the Gauss Jordan Method involves the elimination of the first variable i.e., x from all the equations except the first equation. Then it eliminates the second variable i.e., x2 from all the equations except the second equation and so on proceeding in this manner, finally we eliminate the last variable i.e. in from all the equations except the last equation.²

Sr.No	Gauss Elimination Method	Gauss Jordan Method
1	In this method, the unknowns are eliminated successively and the system is reduced to an upper triangular system from which the unknowns are found by back substitution.	In this method, elimination of unknowns is performed  by all equations not only from equations to follow. Thus the system ultimately reduces to a diagonal matrix form i.e., each equation involving only one unknowns.
2	Finding the solution of n simultaneous linear equation, the number of multiplications and divisions are of the order. n3/3. For Example: if n=5, the number of multiplications and divisions is approximately 42.	Finding the solution of n simultaneous linear equation, the number of multiplications and divisions are of the order. n3/2. For Example: if n=5, the number of multiplications and divisions are approximately 62.
3	It does not seem to be easier but requires about 50 percent fewer operations than Gauss Jordan Method.	It seems to be easier but requires about 50 percent fewer operations than Gauss elimination  Method.
4	For large systems, Gauss Elimination Method is not preferred.	For large systems, Gauss Jordan Method is preferred to Gauss Elimination Method

Why

See Theoretical Knowledge Vs Practical Application.

How

Many of the References and Additional Reading websites, and Videos will assist you with using the Gauss Elimination Method and the Gauss-Jordan Elimination Method.

As some professors say: “It is intuitively obvious to even the most casual observer.”

References

¹ Xoque55. “Why Use Gauss Jordan Elimination Instead Of Gaussian Elimination, Differences”. 2014. Mathematics Stack Exchange. https://math.stackexchange.com/questions/879956/why-use-gauss-jordan-elimination-instead-of-gaussian-elimination-differences.

² “Difference Between Gauss Elimination Method And Gauss Jordan Method | Numerical Method – GeeksforGeeks”. 2021. GeeksforGeeks. https://www.geeksforgeeks.org/difference-between-gauss-elimination-method-and-gauss-jordan-method-numerical-method/.

³ van Aarsen, Klaas. “Why Use Gauss Jordan Elimination Instead Of Gaussian Elimination, Differences”. 2014. Mathematics Stack Exchange. https://math.stackexchange.com/questions/879956/why-use-gauss-jordan-elimination-instead-of-gaussian-elimination-differences.

⁴ The rank of the matrix refers to the number of linearly independent rows or columns in the matrix. ρ(A) is used to denote the rank of matrix A.  A matrix is said to be of rank zero when all of its elements become zero. The rank of the matrix is the dimension of the vector space obtained by its columns. The rank of a matrix cannot exceed more than the number of its rows or columns. The rank of the null matrix is zero.

Additional Reading

Barry, Brett. “The Game Of Gaussian Elimination: An Introduction To Linear Algebra”. 2019. Medium. https://medium.com/i-math/the-game-of-gaussian-elimination-an-introduction-to-linear-algebra-5bcdac63df56.

“Gaussian Elimination”. 2022. CliffsNotes. https://www.cliffsnotes.com/study-guides/algebra/linear-algebra/linear-systems/gaussian-elimination.

“Gauss Elimination Method | Meaning And Solved Example”. 2021. BYJUS. https://byjus.com/maths/gauss-elimination-method/.

“Rank Of A Matrix – Definition | How To Find The Rank Of The Matrix?”. 2022. CueMath. https://www.cuemath.com/algebra/rank-of-a-matrix/.

“Rank Of A Matrix – Formulas. Properties, Examples”. 2022. BYJUS. https://byjus.com/jee/rank-of-a-matrix-and-special-matrices/.

Videos

This precalculus video tutorial provides a basic introduction into the gaussian elimination – a process that involves elementary row operations with 3×3 matrices which allows you to solve a system of linear equations with 3 variables. You need to convert the system of equations into an augmented matrix and use matrix row operations to write it in row echelon form. Next, you can convert back into a system of linear equations and solve using back substitution. This video contains plenty of examples and practice problems.

This seems to be a common misunderstanding with matrices. Echelon form is simply the final matrix after you’ve finished Gauss elimination . (The matrix at 6:10.) It need only have 0’s in the lower left-hand corner. There is no requirement to have 1’s along the main diagonal because they do not help in solving the problem . They are simply wasted operations and are not done in practice. (Unless you want to do Gauss-Jordan elimination… which is also not used in practice.)

This precalculus video tutorial provides a basic introduction into the Gauss Jordan elimination which is a process used to solve a system of linear equations by converting the system into an augmented matrix and using elementary row operations to convert the 3×3 matrix into its reduced row echelon form. You can easily determine the answers once you convert it to that form.

Using Gauss-Jordan elimination to invert a 3×3 matrix.

I was curious to see how the elementary operations geometrically change the planes in each equations. And how put together the algorithm might have a geometric intuition.

I’m on my first semester of engineering and we just learned gaussian elimination. This visual representation has opened my eyes to what I’m really doing when applying the algorithm. thanks a, lot great content!

---

Gaussian Elimination helps to put a matrix in row echelon form, while Gauss-Jordan Elimination puts a matrix in reduced row echelon form. Whatever method we use it gives same answer.

A matrix A of order m * n is said to be in echelon form if it satisfies the following properties
1. All rows consisting of zero only, if exists appear below all non zero rows
2. The first non zero ( leading) element in each row is 1
3. Number of zeros occurring before the first non zero element in each non zero row is greater than the number of such zeros that appear in any preceding row.

Here are 2 rules for row reduced echelon form of a matrix.
Rule 1 Matrix should be in echelon form.
Rule 2 the leading entry in each row is the only non zero element in that column.

Let A be a non zero matrix . If r is the number of non zero rows when it is reduced to row echelon form or to the row reduced echelon form then r is called rank of the matrix.