Advanced Computing Platform for Theoretical Physics

Commit f7dc55a9 authored by Pan Zhang's avatar Pan Zhang
Browse files

a notebook added

parent a163cfaa
%% Cell type:markdown id: tags:
# Different ways of computing gradient of trace of infinite matrix products
$\nabla_T \ln Z = \nabla _T \ln \mathrm{tr}\left[\underbrace{T\times T \times \cdots \times T}_{\mathrm{\infty}}\right]$
We present several methods for computing the gradients with:
- Autograd of $\ln Z$ that is computed using the environment matrix and RG
- Autograd of the leading eigenvalue of the transfer matrix
- Autograd of $\ln Z$ that is computed using the environment vector and power method
- Using cavity of the matrix product
- Using the leading eigenvector
%% Cell type:code id: tags:
``` python
import torch
torch.manual_seed(1)
N = 10 # size of the matrix
A = torch.rand(N, N, dtype=torch.float64)
T = torch.nn.Parameter(A@A.t()) # a symmetric positive definite transfer matrix
```
%% Cell type:markdown id: tags:
### Autograd of $\ln Z$ using environment matrix
Consider a infinite-length chain as product of matrices $T$. Analogous to real-space RG for the one-dimensional Ising model, we merge all pairs of consequtive matrices at each RG step, resulting to a chain with half length (although still infinite), then truncate the spectrum of merged matrices to original dimension for a low-rank approximation. By keep doing this RG step, merged matrices will finally converge to environments (for each tensor $T$) $M$ which alsoindicates that $Z=M\times T \times M$, and $$ \ln Z=\ln(M\times T \times M).$$
%% Cell type:code id: tags:
``` python
lnZ = 0.0
M = T.clone() # M will be the converged envioment tensor
Niter = 20 # number of iterations in RG. That is, 2^Niter matrices will be contracted finally.
for i in range(Niter): # after i steps, there are totally 2^i matrices contracted.
s = M.norm() # This is the normalization of a matrix contracted of 2^i T.
lnZ = lnZ + torch.log(s)/2**i # Notice that we can only record a density of logarithm of the results, for contraction of infinite matrices.
M = M/s
M = M@M
lnZ = lnZ + torch.trace(M)/(2**Niter) # trace(M) is the trace of contraction of all tensors.
lnZ.backward()
RG_grad = T.grad.clone()
```
%% Cell type:markdown id: tags:
### Autograd of the leading eigenvalue of the transfer matrix
In the last section, we have evaluated $\ln Z$ using RG. Actually, this can be evaluated analytically as
$$\ln Z =
\frac{1}{t}\lim_{t\to\infty}\ln \mathrm{tr}(T^t)=\frac{1}{t}\lim_{t\to\infty}\sum_{i=1}^N \lambda_i^t=\lambda_\mathrm{max},$$
where $\lambda_{\mathrm{max}}$ is the leading eigenvalue. Thus we can do back propagation directly on $\lambda_{\mathrm{max}}$.
%% Cell type:code id: tags:
``` python
w, v = torch.symeig(T, eigenvectors=True) # W is an arry of eigenvalues, v is a matrix with each row storing an eigenvector
T.grad.zero_()
lambda_max = torch.log(w[-1])
lambda_max.backward()
eigenvalue_grad = ((T.grad + T.grad.t())/2) # need to symmetrize since it is an upper triangular matrix
```
%% Cell type:markdown id: tags:
### Using cavity of the matrix product
Using the property of the environment matrix $M$, we have $Z=\mathrm{tr}\left[\underbrace{T\times T \times \cdots \times T}_{\mathrm{\infty}}\right]=\mathrm{Tr}(M\times T \times M)$. Then the gradient with respect to a specific $T$ can be written as
$$\nabla_T \ln Z = \frac{M\times M}{\mathrm{tr}(M\times T\times M)}.$$
%% Cell type:code id: tags:
``` python
impurity_grad = (M@M).t()/torch.trace(M@T@M)
```
%% Cell type:markdown id: tags:
### Autograd of $\ln Z$ using vector environment
Here comes the trick: Infinite tensor product with a Periodic Boundary Condition (PBC) is equivalent to that with a Open Boundary Condition (OBC), which is written as
$\ln Z=\ln \left[u\times\underbrace{T\times T \times \cdots \times T}_{\mathrm{\infty}}\times u\right]=\ln(m\times T\times m)$, where $m$ is the environment for the OBC tensor product, a vector. Obviously, $m$ is the leading eigenvector of the transfer matrix $T$ hence is normalized, and the gradient can be written as $$\nabla_T\ln Z=\exp(-\lambda_\mathrm{max}) m\otimes m.$$
%% Cell type:code id: tags:
``` python
v=v[:,-1] # the leading eigenvector of the transfer matrix $T$
eigenvector_grad=v[:,None]@v[None,:]/torch.exp(lambda_max) # outer product of the leading eigenvector and its transpose
```
%% Cell type:markdown id: tags:
### Comparing these gradients
%% Cell type:code id: tags:
``` python
# comparing RG_grad, eigenvector_grad, eigenvalue_grad and impurity_grad
print ((impurity_grad-RG_grad).abs().max().item())
print ((impurity_grad-eigenvalue_grad).abs().max().item())
print ((impurity_grad-eigenvector_grad).abs().max().item())
```
%%%% Output: stream
2.168404344971009e-18
3.469446951953614e-18
4.336808689942018e-18
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment