ALGEBRALinear AlgebraMathematics Calculator
⊙

Hadamard (Element-wise) Product

(A⊙B)[i,j] = A[i,j]·B[i,j]. Same dimensions required. Used in ML (activation masks), signal processing, and entry-wise operations. Distinct from matrix multiplication.

Concept Fundamentals
(A⊙B)[i,j]=aᵢⱼ·bᵢⱼ
Rule
must match
Dimensions
A⊙B or A∘B
Notation
activation masks
ML

Did our AI summary help? Let us know.

A⊙I = I⊙A = A (identity). Different from AB (matrix product). Commutative: A⊙B = B⊙A.

Key quantities
(A⊙B)[i,j]=aᵢⱼ·bᵢⱼ
Rule
Key relation
must match
Dimensions
Key relation
A⊙B or A∘B
Notation
Key relation
activation masks
ML
Key relation

Ready to run the numbers?

Why: Hadamard product appears in neural networks (gating, attention), image processing, and entry-wise scaling.

How: Multiply corresponding elements: (A⊙B)[i,j] = A[i,j] × B[i,j]. Both matrices must be same size.

A⊙I = I⊙A = A (identity).Different from AB (matrix product).
Sources:Wolfram MathWorldKhan Academy

Run the calculator when you are ready.

Hadamard ProductElement-wise multiplication

Hadamard Product Calculator

⟪
⟫
⟨
⟩
⟪
⟫
⟨
⟩

What is the Hadamard Product?

The Hadamard product, also known as the element-wise product or Schur product, is a binary operation that takes two matrices of the same dimensions and produces a matrix where each element is the product of the corresponding elements of the input matrices.

For two matrices A and B of the same size, the Hadamard product A ⊙ B is defined as:

[A∘B]ij=Aij⋅Bij[A \circ B]_{ij} = A_{ij} \cdot B_{ij}

This operation is different from the standard matrix multiplication and can be thought of as a simpler element-by-element multiplication.

Key Properties of the Hadamard Product

Commutativity

For any matrices A and B of the same size:

A∘B=B∘AA \circ B = B \circ A

The order of matrices doesn't affect the result.

Associativity

For matrices A, B, and C of the same size:

(A∘B)∘C=A∘(B∘C)(A \circ B) \circ C = A \circ (B \circ C)

The grouping of operations doesn't affect the result.

Distributivity

For matrices A, B, and C of the same size:

A∘(B+C)=A∘B+A∘CA \circ (B + C) = A \circ B + A \circ C

The Hadamard product distributes over addition.

Scalar Multiplication

For any scalar α and matrices A and B:

α(A∘B)=(αA)∘B=A∘(αB)\alpha(A \circ B) = (\alpha A) \circ B = A \circ (\alpha B)

Scalar multiplication can be factored out of a Hadamard product.

Applications of the Hadamard Product

Signal Processing

In signal processing, the Hadamard product is used for filtering operations. When applying a mask or filter to an image or signal, we're essentially performing an element-wise multiplication of the data with the filter values.

For example, in image processing, applying a gaussian blur involves taking the Hadamard product of a section of an image with a gaussian kernel.

Machine Learning

The Hadamard product is extensively used in various machine learning algorithms:

  • Neural Networks: Used in the backpropagation algorithm for gradient calculations
  • Convolutional Neural Networks: Used in the convolutional layer operations
  • Recurrent Neural Networks: Used in gating mechanisms like LSTM and GRU
  • Attention Mechanisms: Used to apply attention weights to feature vectors

Statistics and Data Analysis

In statistics, the Hadamard product is used for:

  • Computing element-wise variance and covariance
  • Applying weights to data points or features
  • Implementing masked operations where some elements are filtered out
  • Computing correlation matrices from standardized data

Quantum Mechanics

In quantum mechanics and quantum computing:

  • The Hadamard product occurs in the tensor product of quantum states
  • Used in calculating expected values of observables in mixed states
  • Appears in quantum circuit simulations

Special Cases and Examples

Identity Matrix

For any matrix A and the identity matrix I of the same size:

A∘I=(a110⋯0a22⋯⋮⋮⋱)A \circ I = \begin{pmatrix} a_{11} & 0 & \cdots \\ 0 & a_{22} & \cdots \\ \vdots & \vdots & \ddots \end{pmatrix}

The result is a matrix containing only the diagonal elements of A.

All-Ones Matrix

For a matrix A and an all-ones matrix J of the same size:

A∘J=AA \circ J = A

The Hadamard product with an all-ones matrix returns the original matrix.

Zero Matrix

For any matrix A and the zero matrix O of the same size:

A∘O=OA \circ O = O

The result is always the zero matrix.

Mask Matrix

A binary mask matrix M can be used to select specific elements:

A∘M=(a11m11a12m12⋯a21m21a22m22⋯⋮⋮⋱)A \circ M = \begin{pmatrix} a_{11}m_{11} & a_{12}m_{12} & \cdots \\ a_{21}m_{21} & a_{22}m_{22} & \cdots \\ \vdots & \vdots & \ddots \end{pmatrix}

Where mij is either 0 or 1, zeroing out unwanted elements.

Worked Examples

Example 1: Basic Hadamard Product

Calculate the Hadamard product of matrices A and B:

Matrix A:

(1234)\begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}

Matrix B:

(5678)\begin{pmatrix} 5 & 6 \\ 7 & 8 \end{pmatrix}

Step 1: Multiply corresponding elements

[A∘B]11=a11⋅b11=1⋅5=5[A∘B]12=a12⋅b12=2⋅6=12[A∘B]21=a21⋅b21=3⋅7=21[A∘B]22=a22⋅b22=4⋅8=32\begin{align} [A \circ B]_{11} &= a_{11} \cdot b_{11} = 1 \cdot 5 = 5 \\ [A \circ B]_{12} &= a_{12} \cdot b_{12} = 2 \cdot 6 = 12 \\ [A \circ B]_{21} &= a_{21} \cdot b_{21} = 3 \cdot 7 = 21 \\ [A \circ B]_{22} &= a_{22} \cdot b_{22} = 4 \cdot 8 = 32 \end{align}

Step 2: Construct the result matrix

A∘B=(5122132)A \circ B = \begin{pmatrix} 5 & 12 \\ 21 & 32 \end{pmatrix}

The Hadamard product of the matrices is the element-wise multiplication result.

Example 2: Hadamard Product with Identity Matrix

Calculate the Hadamard product of matrix A with the identity matrix:

Matrix A:

(456789101112)\begin{pmatrix} 4 & 5 & 6 \\ 7 & 8 & 9 \\ 10 & 11 & 12 \end{pmatrix}

Identity Matrix:

(100010001)\begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}

Step 1: Multiply corresponding elements

Each element of A is multiplied by the corresponding element in the identity matrix.

Step 2: Construct the result matrix

A∘I=(4000800012)A \circ I = \begin{pmatrix} 4 & 0 & 0 \\ 0 & 8 & 0 \\ 0 & 0 & 12 \end{pmatrix}

The result is a diagonal matrix containing only the diagonal elements of the original matrix.

Historical Context and Computational Considerations

Historical Development

The Hadamard product is named after the French mathematician Jacques Hadamard (1865-1963), who made significant contributions to functional analysis and matrix theory. The operation is also sometimes called the Schur product, after Issai Schur, who studied its properties in relation to positive definite matrices.

While element-wise multiplication seems straightforward today, its formal properties and applications in matrix theory weren't fully developed until the early 20th century.

Computational Efficiency

The Hadamard product is computationally efficient, with a time complexity of O(n²) for n×n matrices, which is better than the O(n³) complexity of standard matrix multiplication. Each element computation is independent, making it highly parallelizable.

Modern computing libraries like NumPy, TensorFlow, and PyTorch optimize Hadamard product operations for performance across different hardware including CPUs, GPUs, and TPUs.

Relation to Other Operations

The Hadamard product has interesting connections to other matrix operations:

  • It's a subcase of the Kronecker product when applied to vectors
  • For vectors, it's related to the inner product through the identity: ⟨a ∘ b, 1⟩ = ⟨a, b⟩
  • The Frobenius inner product of matrices can be computed using the Hadamard product: ⟨A, B⟩ = sum(A ∘ B)
  • In information theory, it's related to mutual information matrices

For educational and informational purposes only. Verify with a qualified professional.

🧮 Fascinating Math Facts

⊙

A⊙B = B⊙A (commutative)

📐

(A⊙B)⊙C = A⊙(B⊙C)

👈 START HERE
⬅️Jump in and explore the concept!
AI

Related Calculators

Tensor Product Calculator

Tensor Product Calculator - Calculate and learn about linear-algebra concepts

Mathematics

SVD Calculator

S V D Calculator - Calculate and learn about linear-algebra concepts

Mathematics

Pseudoinverse Calculator

Pseudoinverse Calculator - Calculate and learn about linear-algebra concepts

Mathematics

L U Decomposition Calculator

L U Decomposition Calculator - Calculate and learn about linear-algebra concepts

Mathematics

Q R Decomposition Calculator

Q R Decomposition Calculator - Calculate and learn about linear-algebra concepts

Mathematics

Adjoint Matrix Calculator

Adjoint Matrix Calculator - Calculate and learn about linear-algebra concepts

Mathematics