I have a simple "Neural Network" that takes two inputs and computes a linear combination and outputs a scalar. In the case of one single equation, i.e., in 1D, we would have: weight1*inputA + weight2*inputB = output. Now, I am interested in multi-dimensional inputs, i.e., that inputA is a vector instead. So I want my network to apply the weight1 to a vector, for example: weight1 * [input1, input2] + weight2 * [input3, input4]. In this setting, I would want the output to be a vector too: [out1, out2], where out1 = weight1*inputA + weight2*inputB. However, I don't want to change the input and output dimensions of my network. As in the 1D case, I would initialise the network as net = LinearNet(2,1), since we take two inputs, inputA and inputB, and get one output. I understand that the input and output itself are multi-dimensional, but this should not bother my network.
Below is a min. working example:
import numpy as np
import torch
from torch import nn
def f(x,a,b,c,d):
x_next = np.zeros((2,))
x_next[0] = x[0]*(a-b*x[1])
x_next[1] = -x[1]*(c-d*x[0])
return x_next #returns [2,:]
a = 1
b = 0.5
c = 1
d = 0.5
x01 = 1 #init cond. varA
x02 = 2 #init cond. varB
params = [a,b,c,d,x01,x02]
# ==================
h = 0.001
T = 10
K = int(T/h)
# forward euler approx.
x_traj_FE = np.zeros((2,K))
a, b, c, d, x_traj_FE[0,0], x_traj_FE[1,0] = params
for k in range(K-1):
x_traj_FE[:,k+1] = x_traj_FE[:,k] + h*f(x_traj_FE[:,k],a,b,c,d)
# ==================
class LinearNet(nn.Module):
def __init__(self, input_dim, output_dim):
super(LinearNet, self).__init__()
self.layer1 = nn.Linear(input_dim, output_dim, bias=False)
#no activation function
def forward(self, x):
x = self.layer1(x)
#no activation
return x
torch.manual_seed(789)
net = LinearNet(2,1)
x_traj = np.zeros((2,K))
a, b, c, d, x_traj[0,0], x_traj[1,0] = params
for k in range(K-1):
print(net.layer1.weight)
input1 = x_traj[:,k]
input2 = f(x_traj[:,k],a,b,c,d)
inputs = torch.Tensor(np.array([input1, input2]))
print('input: '+str(inputs))
print('shape: '+str(inputs.shape))
print('output: '+str(net(inputs)))
break
If we run this, we will receive the following output:
Parameter containing:
tensor([[0.3871, 0.5595]], requires_grad=True)
input: tensor([[ 1., 2.],
[ 0., -1.]])
shape: torch.Size([2, 2])
output: tensor([[ 1.5060],
[-0.5595]], grad_fn=<MmBackward0>)
Eventually, I would like to compute one forward euler step of the coupled ode f, where the input1 is the initial conditions of each equation within the system f, and input2 is the system evaluated at the previous step. If we do it manually, one step can be calculated as (given the weights from the network above):
weights = [0.3871, 0.5595]
x_traj = np.zeros((2,K))
a, b, c, d, x_traj[0,0], x_traj[1,0] = params
for k in range(K-1):
x_traj[:,k+1] = weights[0]*x_traj[:,k] + weights[1]*f(x_traj[:,k],a,b,c,d)
break
x_traj
And the output is:
array([[1. , 0.3871, 0. , ..., 0. , 0. , 0. ],
[2. , 0.2147, 0. , ..., 0. , 0. , 0. ]])
As we see, the output of the network differs from the manual computation. I don't see how the network computes this scalar-vector multiplication and hence, I can't understand how to retrieve the same output as with the manual computation.