Questions tagged [tensor]

In machine learning, tensor is a multidimensional (multi-index, or multi-way) array of numbers, i.e. a generalization of a matrix.

Filter by
Sorted by
Tagged with
0 votes
0 answers
68 views

Difference between multi-head and single-head attention

Attention, as long as gradient calculations care, is two nested tensor multiplications and a softmax. I thought that, then, multi-head attention with $h=8$ and $d_k=64$ results in the same tensor with ...
tolgarecep's user avatar
0 votes
0 answers
8 views

How to adjust the scaling of the new data while use Incremental training of a neural network?

I am planning to use incremental training of my neural network model since I continually get new data and at present retrain the model after a period of time but the training window shifts forward. To ...
user62198's user avatar
  • 109
2 votes
1 answer
75 views

mgcv: Use of s() or te() with interactions in GAMs?

I am trying to model CO2 fluxes (fco2) using a number of environmental parameters using a GAM in mgcv. Specifically, I have leaf temperature (tl), vapour pressure deficit (vpd), and soil water content ...
J-M's user avatar
  • 23
0 votes
0 answers
27 views

Downsampling Issue with Conv2D Layer in AWS SageMaker Notebook

I am encountering an issue with the downsampling in the Conv2D layer when running my code on an AWS SageMaker Notebook. The code runs fine on my local machine and in Google Colab, but I am facing this ...
batmanbeginscricket's user avatar
0 votes
0 answers
46 views

Constaint on te() tensor product gam mgcv

In the mgcv package in R, I'm working on models whose covariates are forced to change shape at the median (=0). These are the models: ...
Tesla's user avatar
  • 1
1 vote
0 answers
19 views

Better default prior for non-negative canonical polyadic decomposition of counts than Exp(1)?

Suppose I have a instance of a random $k$-mode tensor $X_{n_1 \times \ldots \times n_k}$ of count data. I would like to perform non-negative canonical polyadic decomposition of this tensor using ...
Galen's user avatar
  • 7,008
0 votes
1 answer
43 views

What "Convolution filters along the time axis" means?

Suppose that I have a tensor of height:25 and width:50. Height is my temporal axis, therefore I have a window of 25 time steps. Therefore my input tensor is: I want to extract temporal features / ...
Mas A's user avatar
  • 171
0 votes
1 answer
212 views

bos_token for a custom Transformer

I am trying to use a Transformer to solve a time-series problem. I built the model using the Pytorch library. And I am planning to train the model from scratch. The model is looking back last L time-...
Wenuka's user avatar
  • 113
4 votes
1 answer
58 views

Einstein notation $-$ or another $-$ to denote constraints in high dimensional ILP problems

When discussing marginal sums of arrays in 3 dimensions or more, is it customary in the statistical and/or data science communities to use the Einstein summation convention? Is some other form ...
Peter Leopold's user avatar
1 vote
2 answers
67 views

What does Cayley's hyperdeterminant of a 2x2x2 mixed-product moment tensor tell us about how two variables are related?

Suppose we have a collection of random variables $S = \{ X_0, X_1 \}$ encoded into the $2 \times 2 \times 2$ tensor $$\mathcal{C}[i, j, k] = \mathbb{E}[X_i X_j X_k]$$ where $X_i, X_j, X_k \in S$ and $...
Galen's user avatar
  • 7,008
1 vote
0 answers
55 views

Is there a multilinear kernel principal components analysis?

PCA can be extended to kPCA using the kernel trick. MPCA is a multilinear extension of PCA that involves multiple matrices for the different modes of the data tensor. Can MCPA be similarly extended ...
Galen's user avatar
  • 7,008
1 vote
0 answers
27 views

Is there a multilinear principal component regression?

PCR is a linear regression problem on top of PCA. Analogically, is there a 'multilinear PCR' as a linear or multilinear regression problem (e.g. CP tensor regression and Tucker tensor regression for ...
Galen's user avatar
  • 7,008
1 vote
1 answer
168 views

What is multilinear principal components analysis?

I've gotten a lot of usage out of principal component analysis, and after recently learning the basics of performing canonical polyadic decomposition I was intrigued to learn that there exists a ...
Galen's user avatar
  • 7,008
1 vote
0 answers
27 views

Is there an improved canonical polyadic decomposition for symmetric tensors?

Let us suppose I want to find a CP decomposition of a $n$-mode tensor $\mathcal{A}$. Fortunately the tensor has the permutation symmetry $$\mathcal{A}[i_1, \cdots, i_n] = \mathcal{A}[\sigma (i_1), \...
Galen's user avatar
  • 7,008
1 vote
0 answers
73 views

How to select ranks to search for CP decomposition of 4-mode tensor of experimental gene expression data?

I have a sparse (0.625 percent non-zero occupancy) data tensor with shape (118, 16, 5009, 10). I would like to try exploring the data using CP decomposition. The ...
Galen's user avatar
  • 7,008
1 vote
1 answer
149 views

Presenting 2D smoother of GAM

I had a look around and couldn't find the answer to my problem, so hopefully this is a new question. I tried to fit a GAM with two continuous explanatory variables, one of them is Day of the year and ...
Toastershock91's user avatar
1 vote
0 answers
14 views

CP Tensor Decomposition and Correlating Sample Magnitudes with Variables of Interest

I am learning about tensor decomposition, specifically CP, and am trying to understand if I can use it for my research. To give a bit more detail, I have brain imaging data from 10 participants, with ...
pdhami's user avatar
  • 335
2 votes
1 answer
826 views

Mathematical representation of 1D convolution

How does one write the mathematical formula for conv1d used in PyTorch, including parameters like stride length and padding? For instance, I can write ...
Kevin's user avatar
  • 73
1 vote
1 answer
124 views

How to express the notion of a vector, where every element inside is matrix?

I met a question when I tried to express a vector, where every element inside is a matrix. We know the notion usually works like this: scalar: $a$ vector: $\boldsymbol{a}$ matrix: $\boldsymbol A$ ...
Nick Nick Nick's user avatar
1 vote
1 answer
325 views

R/mgcv: How do you interpret the 'fixed effects' for multivariate te() tensor products in the 'lme' part of a gamm model in R?

When fitting a 'gamm' model in the R package mgcv, and using a te() tensor product of three variables, the lme part of the model reports seven fixed effects, from '...
Alvaro Rucua's user avatar
1 vote
0 answers
18 views

Are the arrows connecting nodes called as Tensors?

I asked a close friend of mine to explain Neural Networks at high level. At one point, he explained to me the arrows that connecting nodes are called as Tensors and that each tensor has a weight. I ...
jeffbRTC's user avatar
  • 111
0 votes
1 answer
1k views

Normalized 2D tensor values are not in range 0-1

Below function takes in 2D tensor and normalizes it using broadcasting .The issue is except all values to be in range 0-1 but the result has values outside this range . How to get all values in 2D ...
star's user avatar
  • 135
1 vote
0 answers
358 views

Tensor linearization interview question [closed]

I got the following question in a coding interview for machine learning engineer position. Write a function: ...
Encipher's user avatar
  • 155
1 vote
0 answers
23 views

What are the basic statistics to describe tensor data distribution [duplicate]

For numeric data, the basic/straightforward way to describe its distribution is to use some metrics like mean, min, max, std et al. What about tensor data (vectors and high dimensional data)? Of ...
cxs1031's user avatar
  • 31
1 vote
1 answer
385 views

Reverse-Mode Automatic Differentiation with respect to a Matrix: How to "Matrix Multiply" 4D Tensors?

This is a follow up question I have on this excellent answer: https://stats.stackexchange.com/a/235758/307400. I will save me writing down any details about reverse-mode automatic differentiation, the ...
cherrywoods's user avatar
2 votes
1 answer
479 views

How can we perform matrix factorisation for three dimensional matrix?

I am working on the recommendation system in which I have three factors user_id, time and ...
Lavanya Sagunthala's user avatar
5 votes
1 answer
97 views

obtaining 4th moment tensor under change of coordinates

Suppose I have a random real-valued vector $x=x_1,\ldots,x_d$ and $M_{ijkl}=E[x_i x_j x_k x_l]$ and apply a change of coordinates $y=Ax$ where $A$ is an orthogonal matrix. How do I obtain $N_{ijkl}=E[...
Yaroslav Bulatov's user avatar
1 vote
0 answers
93 views

How to find eigenvalues and eigenvectors of the cokurtosis matrix?

Kurtosis is the fourth statistical moment of a random variable's distribution. Unlike the variance-covariance matrix $\Sigma$, which had a shape of $p\times p$, the kurtosis-cokurtosis matrix is ...
develarist's user avatar
  • 3,481
2 votes
0 answers
53 views

Tensor product between an ispline and a bspline for fitting data that should be monotonic in one dimension

I'm not very familiar with the process for solving tensor product basis fittings. I've done some work with fitting an ispline basis with a non-negative-least-squares solver to fit a monotonic spline ...
Joey's user avatar
  • 21
1 vote
1 answer
74 views

What is the actual use of GAN model? Is it only used to generate the data that closely resembles original dataset?

I am very new to tensor flow. I came across GANS. From what I understand in GANs there are 2 models, Generator and Discriminator. Generator job is to generate the data that will be able to fool the ...
Sam's user avatar
  • 113
0 votes
0 answers
23 views

How to derive mathematically that derivative of |Ax-y|^2 with respect to A is 2|Ax-y| x^T [duplicate]

How to get transpose part when derive mathematically $$ \frac {\partial|Ax-y|^2}{\partial A} = 2|Ax-y|x^T $$
Rishi Jaiswal's user avatar
0 votes
2 answers
154 views

Understandable way of thinking about higher order tensors

I study deep learning and the one of the major problem I face is I can't imagine shape of higher order tensors in my head. for instance - A 2d tensor - (x,y) is a rectangle with x,y along its length ...
Beginner's user avatar
  • 103
1 vote
0 answers
383 views

Compact/Vectorized Multiclass Logistic Regression Hessian

I know that the Hessian of the categorical cross entropy w.r.t the weights is given by $$\frac{\partial^2 L}{\partial w^2} = \sum_{i=1}^{m} (Diag(\hat{y}_i)-\hat{y}_i^T \hat{y}_i) \otimes x_i^T x_i$$ ...
nubol23's user avatar
  • 33
10 votes
2 answers
1k views

"row" and "column" are the names of axes of 2d array, is there a similar naming for a 3d array?

row and column are the names of axes of 2d array. this python array, array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) could be viewed as a matrix that ...
whnlp's user avatar
  • 227
2 votes
0 answers
67 views

Why are some robust algorithms valid for Tucker decomposition, but not for CP decomposition?

I have been reading up about CP and Tucker decomposition. It makes sense that CP decomposition is a special case of Tucker decomposition, where the core tensor is super-diagonal. However, if this is ...
Roy's user avatar
  • 121
2 votes
0 answers
213 views

Diffusion tensor as a covariance matrix

TLDR: In nuclear magnetic resonance (NMR), to study molecular diffusion we assume that molecules displace in 3D space according to a trivariate gaussian distribution. The variables are then the ...
user241848's user avatar
4 votes
1 answer
622 views

Higher moments of linear regression residuals?

I previously asked this on Math StackExchange, with no success, but this post will add to that with some simulations. Background In the following linear regression with i.i.d $\epsilon_i$ $(i = 1, \...
Tom Chen's user avatar
  • 591
2 votes
0 answers
312 views

Regarding the quantics tensor train (QTT) format

I originally posted this question in Data Science Stack Exchange, however, I think this forum may be better for this question. I believe I have a fair understanding of the tensor train (TT) format, ...
grad_student's user avatar
0 votes
0 answers
505 views

Random forest for tensors

Say we have have input tensor $X \in \mathbb{R}^{T \times N \times P}$ and output tensor $Y \in \mathbb{R}^{T \times N \times K}$, and we aim to build a Random Forests model $Y = f(X)+ \epsilon$. ...
adam's user avatar
  • 115
2 votes
1 answer
2k views

Can a Fully Connected layer transform a 4D tensor to a 3D tensor by itself?

Recently, I was researching some topics in biometrics and I stumbled upon this paper. They have a table there (Table 1) in which they state that they used a modified CNN from this paper (Table 9). In ...
Colonder's user avatar
  • 125
3 votes
1 answer
7k views

Machine learning methods for multi-dimensional input and output

I have a large dataset where my input is an $M$-dimensional tensor, and each input has a corresponding $N$-dimensional output. My goal is to train a method to learn outputs from the millions of inputs ...
Mathews24's user avatar
  • 469
2 votes
0 answers
80 views

Is there any sort of quadratic SVD for dimensionality reduction?

X-Posted on math.stackexchange, apologies, though I thought this was equally relevant to both communities. I'm wondering if there exists any higher-order SVD for dimensionality reduction. Note that ...
user650261's user avatar
0 votes
1 answer
126 views

Parameters in a neural tensor network

I am reading the paper of "Reasoning With Neural Tensor Networks for Knowledge Base Completion". I read it many times but I couldn't understand the parameters that are used especially the parameter U. ...
bttX's user avatar
  • 3
0 votes
1 answer
481 views

Best way to represent 3D data for Neural Networks

I want to train a generative model over a dataset where each example is a $X = (N,3)$ matrix representing $N$ points in $\mathbb{R}^3$. The local structure (i.e. the correlations between neighboring ...
Toool's user avatar
  • 155
1 vote
1 answer
196 views

Satellite data pre-processing for Keras CNNs [closed]

I’m looking at satellite data and want to do object detection using CNNs in Keras. I’m currently pre-processing the data (turning them into tensors) that I’ve obtained which include the original ...
FinanceGardener's user avatar
0 votes
1 answer
218 views

Shape of a tensor [closed]

Suppose I have a variable that looks like this [[[1., 2., 3.]], [[7., 8., 9.]]] I have read this is a rank 3 tensor with shape ...
iratelilkid's user avatar
0 votes
0 answers
84 views

References on tensor algebra for machine learning? [duplicate]

So tensors come up a lot in machine learning in various settings. Any math major will have studied linear algebra heavily and it's usually much simpler to use linear operators and the corresponding ...
dat's user avatar
  • 1
2 votes
3 answers
442 views

Tensor Classification Models

Aside from Convolution Neural Networks, are there any other methods that allow for classification of Tensors? My observations consist of multi-dimensional tensors with height of 1, where each channel ...
mamafoku's user avatar
  • 211
2 votes
1 answer
1k views

CP decomposition for tensor factorization

I am trying to understand CP decomposition for a three way tensor. Lets have a tensor which has the dimensions I by J by K. When we apply CP decomposition, it decomposes the tensor as a sum of a rank-...
mgokhanbakal's user avatar
0 votes
1 answer
505 views

Fitting Tensor Product P-splines, Penalty Parameters

I am working with the mgcv package in r and I am fitting tensor product P-splines. ...
mprice's user avatar
  • 43