Questions tagged [tensor]
In machine learning, tensor is a multidimensional (multi-index, or multi-way) array of numbers, i.e. a generalization of a matrix.
64
questions
0
votes
0
answers
68
views
Difference between multi-head and single-head attention
Attention, as long as gradient calculations care, is two nested tensor multiplications and a softmax. I thought that, then, multi-head attention with $h=8$ and $d_k=64$ results in the same tensor with ...
0
votes
0
answers
8
views
How to adjust the scaling of the new data while use Incremental training of a neural network?
I am planning to use incremental training of my neural network model since I continually get new data and at present retrain the model after a period of time but the training window shifts forward. To ...
- 109
2
votes
1
answer
75
views
mgcv: Use of s() or te() with interactions in GAMs?
I am trying to model CO2 fluxes (fco2) using a number of environmental parameters using a GAM in mgcv.
Specifically, I have leaf temperature (tl), vapour pressure deficit (vpd), and soil water content ...
- 23
0
votes
0
answers
27
views
Downsampling Issue with Conv2D Layer in AWS SageMaker Notebook
I am encountering an issue with the downsampling in the Conv2D layer when running my code on an AWS SageMaker Notebook. The code runs fine on my local machine and in Google Colab, but I am facing this ...
0
votes
0
answers
46
views
Constaint on te() tensor product gam mgcv
In the mgcv package in R, I'm working on models whose covariates are forced to change shape at the median (=0). These are the models:
...
- 1
1
vote
0
answers
19
views
Better default prior for non-negative canonical polyadic decomposition of counts than Exp(1)?
Suppose I have a instance of a random $k$-mode tensor $X_{n_1 \times \ldots \times n_k}$ of count data. I would like to perform non-negative canonical polyadic decomposition of this tensor using ...
- 7,008
0
votes
1
answer
43
views
What "Convolution filters along the time axis" means?
Suppose that I have a tensor of height:25 and width:50. Height is my temporal axis, therefore I have a window of 25 time steps. Therefore my input tensor is:
I want to extract temporal features / ...
- 171
0
votes
1
answer
212
views
bos_token for a custom Transformer
I am trying to use a Transformer to solve a time-series problem. I built the model using the Pytorch library. And I am planning to train the model from scratch. The model is looking back last L time-...
- 113
4
votes
1
answer
58
views
Einstein notation $-$ or another $-$ to denote constraints in high dimensional ILP problems
When discussing marginal sums of arrays in 3 dimensions or more, is it customary in the statistical and/or data science communities to use the Einstein summation convention? Is some other form ...
- 2,194
1
vote
2
answers
67
views
What does Cayley's hyperdeterminant of a 2x2x2 mixed-product moment tensor tell us about how two variables are related?
Suppose we have a collection of random variables $S = \{ X_0, X_1 \}$ encoded into the $2 \times 2 \times 2$ tensor
$$\mathcal{C}[i, j, k] = \mathbb{E}[X_i X_j X_k]$$
where $X_i, X_j, X_k \in S$ and $...
- 7,008
1
vote
0
answers
55
views
Is there a multilinear kernel principal components analysis?
PCA can be extended to kPCA using the kernel trick.
MPCA is a multilinear extension of PCA that involves multiple matrices for the different modes of the data tensor.
Can MCPA be similarly extended ...
- 7,008
1
vote
0
answers
27
views
Is there a multilinear principal component regression?
PCR is a linear regression problem on top of PCA.
Analogically, is there a 'multilinear PCR' as a linear or multilinear regression problem (e.g. CP tensor regression and Tucker tensor regression for ...
- 7,008
1
vote
1
answer
168
views
What is multilinear principal components analysis?
I've gotten a lot of usage out of principal component analysis, and after recently learning the basics of performing canonical polyadic decomposition I was intrigued to learn that there exists a ...
- 7,008
1
vote
0
answers
27
views
Is there an improved canonical polyadic decomposition for symmetric tensors?
Let us suppose I want to find a CP decomposition of a $n$-mode tensor $\mathcal{A}$. Fortunately the tensor has the permutation symmetry
$$\mathcal{A}[i_1, \cdots, i_n] = \mathcal{A}[\sigma (i_1), \...
- 7,008
1
vote
0
answers
73
views
How to select ranks to search for CP decomposition of 4-mode tensor of experimental gene expression data?
I have a sparse (0.625 percent non-zero occupancy) data tensor with shape (118, 16, 5009, 10). I would like to try exploring the data using CP decomposition. The ...
- 7,008
1
vote
1
answer
149
views
Presenting 2D smoother of GAM
I had a look around and couldn't find the answer to my problem, so hopefully this is a new question.
I tried to fit a GAM with two continuous explanatory variables, one of them is Day of the year and ...
1
vote
0
answers
14
views
CP Tensor Decomposition and Correlating Sample Magnitudes with Variables of Interest
I am learning about tensor decomposition, specifically CP, and am trying to understand if I can use it for my research.
To give a bit more detail, I have brain imaging data from 10 participants, with ...
- 335
2
votes
1
answer
826
views
Mathematical representation of 1D convolution
How does one write the mathematical formula for conv1d used in PyTorch, including parameters like stride length and padding?
For instance, I can write
...
- 73
1
vote
1
answer
124
views
How to express the notion of a vector, where every element inside is matrix?
I met a question when I tried to express a vector, where every element inside is a matrix. We know the notion usually works like this:
scalar: $a$
vector: $\boldsymbol{a}$
matrix: $\boldsymbol A$
...
- 133
1
vote
1
answer
325
views
R/mgcv: How do you interpret the 'fixed effects' for multivariate te() tensor products in the 'lme' part of a gamm model in R?
When fitting a 'gamm' model in the R package mgcv, and using a te() tensor product of three variables, the lme part of the model reports seven fixed effects, from '...
- 13
1
vote
0
answers
18
views
Are the arrows connecting nodes called as Tensors?
I asked a close friend of mine to explain Neural Networks at high level. At one point, he explained to me the arrows that connecting nodes are called as Tensors and that each tensor has a weight.
I ...
- 111
0
votes
1
answer
1k
views
Normalized 2D tensor values are not in range 0-1
Below function takes in 2D tensor and normalizes it using broadcasting .The issue is except all values to be in range 0-1 but the result has values outside this range . How to get all values in 2D ...
- 135
1
vote
0
answers
358
views
Tensor linearization interview question [closed]
I got the following question in a coding interview for machine learning engineer position.
Write a function:
...
- 155
1
vote
0
answers
23
views
What are the basic statistics to describe tensor data distribution [duplicate]
For numeric data, the basic/straightforward way to describe its distribution is to use some metrics like mean, min, max, std et al.
What about tensor data (vectors and high dimensional data)?
Of ...
- 31
1
vote
1
answer
385
views
Reverse-Mode Automatic Differentiation with respect to a Matrix: How to "Matrix Multiply" 4D Tensors?
This is a follow up question I have on this excellent answer: https://stats.stackexchange.com/a/235758/307400. I will save me writing down any details about reverse-mode automatic differentiation, the ...
- 111
2
votes
1
answer
479
views
How can we perform matrix factorisation for three dimensional matrix?
I am working on the recommendation system in which I have three factors user_id, time and ...
5
votes
1
answer
97
views
obtaining 4th moment tensor under change of coordinates
Suppose I have a random real-valued vector $x=x_1,\ldots,x_d$ and $M_{ijkl}=E[x_i x_j x_k x_l]$ and apply a change of coordinates $y=Ax$ where $A$ is an orthogonal matrix. How do I obtain $N_{ijkl}=E[...
- 6,107
1
vote
0
answers
93
views
How to find eigenvalues and eigenvectors of the cokurtosis matrix?
Kurtosis is the fourth statistical moment of a random variable's distribution. Unlike the variance-covariance matrix $\Sigma$, which had a shape of $p\times p$, the kurtosis-cokurtosis matrix is ...
- 3,481
2
votes
0
answers
53
views
Tensor product between an ispline and a bspline for fitting data that should be monotonic in one dimension
I'm not very familiar with the process for solving tensor product basis fittings. I've done some work with fitting an ispline basis with a non-negative-least-squares solver to fit a monotonic spline ...
- 21
1
vote
1
answer
74
views
What is the actual use of GAN model? Is it only used to generate the data that closely resembles original dataset?
I am very new to tensor flow. I came across GANS. From what I understand in GANs there are 2 models, Generator and Discriminator. Generator job is to generate the data that will be able to fool the ...
- 113
0
votes
0
answers
23
views
How to derive mathematically that derivative of |Ax-y|^2 with respect to A is 2|Ax-y| x^T [duplicate]
How to get transpose part when derive mathematically $$ \frac {\partial|Ax-y|^2}{\partial A} = 2|Ax-y|x^T $$
0
votes
2
answers
154
views
Understandable way of thinking about higher order tensors
I study deep learning and the one of the major problem I face is I can't imagine shape of higher order tensors in my head.
for instance -
A 2d tensor - (x,y) is a rectangle with x,y along its length ...
- 103
1
vote
0
answers
383
views
Compact/Vectorized Multiclass Logistic Regression Hessian
I know that the Hessian of the categorical cross entropy w.r.t the weights is given by
$$\frac{\partial^2 L}{\partial w^2} = \sum_{i=1}^{m} (Diag(\hat{y}_i)-\hat{y}_i^T \hat{y}_i) \otimes x_i^T x_i$$
...
- 33
10
votes
2
answers
1k
views
"row" and "column" are the names of axes of 2d array, is there a similar naming for a 3d array?
row and column are the names of axes of 2d array.
this python array,
array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
could be viewed as a matrix that ...
- 227
2
votes
0
answers
67
views
Why are some robust algorithms valid for Tucker decomposition, but not for CP decomposition?
I have been reading up about CP and Tucker decomposition.
It makes sense that CP decomposition is a special case of Tucker decomposition, where the core tensor is super-diagonal. However, if this is ...
- 121
2
votes
0
answers
213
views
Diffusion tensor as a covariance matrix
TLDR: In nuclear magnetic resonance (NMR), to study molecular diffusion we assume that molecules displace in 3D space according to a trivariate gaussian distribution. The variables are then the ...
- 21
4
votes
1
answer
622
views
Higher moments of linear regression residuals?
I previously asked this on Math StackExchange, with no success, but this post will add to that with some simulations.
Background
In the following linear regression with i.i.d $\epsilon_i$ $(i = 1, \...
- 591
2
votes
0
answers
312
views
Regarding the quantics tensor train (QTT) format
I originally posted this question in Data Science Stack Exchange, however, I think this forum may be better for this question.
I believe I have a fair understanding of the tensor train (TT) format, ...
- 21
0
votes
0
answers
505
views
Random forest for tensors
Say we have have input tensor $X \in \mathbb{R}^{T \times N \times P}$ and output tensor $Y \in \mathbb{R}^{T \times N \times K}$, and we aim to build a Random Forests model $Y = f(X)+ \epsilon$.
...
- 115
2
votes
1
answer
2k
views
Can a Fully Connected layer transform a 4D tensor to a 3D tensor by itself?
Recently, I was researching some topics in biometrics and I stumbled upon this paper. They have a table there (Table 1) in which they state that they used a modified CNN from this paper (Table 9). In ...
- 125
3
votes
1
answer
7k
views
Machine learning methods for multi-dimensional input and output
I have a large dataset where my input is an $M$-dimensional tensor, and each input has a corresponding $N$-dimensional output. My goal is to train a method to learn outputs from the millions of inputs ...
- 469
2
votes
0
answers
80
views
Is there any sort of quadratic SVD for dimensionality reduction?
X-Posted on math.stackexchange, apologies, though I thought this was equally relevant to both communities.
I'm wondering if there exists any higher-order SVD for dimensionality reduction. Note that ...
- 121
0
votes
1
answer
126
views
Parameters in a neural tensor network
I am reading the paper of "Reasoning With Neural Tensor Networks for Knowledge Base Completion". I read it many times but I couldn't understand the parameters that are used especially the parameter U. ...
- 3
0
votes
1
answer
481
views
Best way to represent 3D data for Neural Networks
I want to train a generative model over a dataset where each example is a $X = (N,3)$ matrix representing $N$ points in $\mathbb{R}^3$.
The local structure (i.e. the correlations between neighboring ...
- 155
1
vote
1
answer
196
views
Satellite data pre-processing for Keras CNNs [closed]
I’m looking at satellite data and want to do object detection using CNNs in Keras. I’m currently pre-processing the data (turning them into tensors) that I’ve obtained which include the original ...
0
votes
1
answer
218
views
Shape of a tensor [closed]
Suppose I have a variable that looks like this
[[[1., 2., 3.]], [[7., 8., 9.]]]
I have read this is a rank 3 tensor with shape ...
- 91
0
votes
0
answers
84
views
References on tensor algebra for machine learning? [duplicate]
So tensors come up a lot in machine learning in various settings. Any math major will have studied linear algebra heavily and it's usually much simpler to use linear operators and the corresponding ...
- 1
2
votes
3
answers
442
views
Tensor Classification Models
Aside from Convolution Neural Networks, are there any other methods that allow for classification of Tensors? My observations consist of multi-dimensional tensors with height of 1, where each channel ...
- 211
2
votes
1
answer
1k
views
CP decomposition for tensor factorization
I am trying to understand CP decomposition for a three way tensor. Lets have a tensor which has the dimensions I by J by K. When we apply CP decomposition, it decomposes the tensor as a sum of a rank-...
- 105
0
votes
1
answer
505
views
Fitting Tensor Product P-splines, Penalty Parameters
I am working with the mgcv package in r and I am fitting tensor product P-splines.
...
- 43