Skip to main content

Questions tagged [tensor]

In machine learning, tensor is a multidimensional (multi-index, or multi-way) array of numbers, i.e. a generalization of a matrix.

Filter by
Sorted by
Tagged with
1 vote
0 answers
22 views

How to leverage the separable functions in MCMC sampling? [closed]

I'm considering the posterior of a parametric model via the Bayesian approach. More specificity, I have a parametric model $u(p_1,p_2, p_3) = u_1(p_1) \times u_2(p_2) \times u_3(p_3)$ and I want to ...
CC Kuo's user avatar
  • 11
0 votes
1 answer
60 views

Is the design matrix in a panel regression model a tensor?

In a panel regression model of the form $$Y_{it} = \mathbf{X}_{it} \pmb{\beta} + \epsilon_{it}$$ where $Y_{it}$ is the dependent variable for unit $i$ at time $t$ $\mathbf{X}_{it}$ is a vector of $K$ ...
Peter Jordanson's user avatar
2 votes
0 answers
12 views

Is spiked tensor decomposition a special case of INDSCAL decomposition?

I understand that "Spiked" often refers to the presence of a dominant component (or a few dominant components) in a tensor decomposition. Spiked tensor decomposition is applied to multi-way ...
Omar Shehab's user avatar
0 votes
1 answer
71 views

Transformers: Cross Attention Tensor Shapes During Inference Mode

Using the "classic" transformer model describing in "Attention is All You Need", I'm struggling to understand how the Encoder output is used by the Decoder during cross attention ...
NickBraunagel's user avatar
0 votes
0 answers
49 views

How do continuous partial derivatives depend on $n$ in maximum likelihood estimation?

I'm reading Tensor Methods in Statistics by McCullagh 1987, (P209 for this question) and I can't understand one step he uses. He begins with the usual log-likelihood \begin{equation*} l(\theta; Y) =...
Nick Green's user avatar
0 votes
1 answer
343 views

Difference between multi-head and single-head attention

Attention, as long as gradient calculations care, is two nested tensor multiplications and a softmax. I thought that, then, multi-head attention with $h=8$ and $d_k=64$ results in the same tensor with ...
tolgarecep's user avatar
0 votes
0 answers
16 views

How to adjust the scaling of the new data while use Incremental training of a neural network?

I am planning to use incremental training of my neural network model since I continually get new data and at present retrain the model after a period of time but the training window shifts forward. To ...
user62198's user avatar
  • 109
2 votes
1 answer
577 views

mgcv: Use of s() or te() with interactions in GAMs?

I am trying to model CO2 fluxes (fco2) using a number of environmental parameters using a GAM in mgcv. Specifically, I have leaf temperature (tl), vapour pressure deficit (vpd), and soil water content ...
J-M's user avatar
  • 23
1 vote
1 answer
81 views

Constaint on te() tensor product gam mgcv

In the mgcv package in R, I'm working on models whose covariates are forced to change shape at the median (=0). These are the models: ...
Tesla's user avatar
  • 11
1 vote
0 answers
20 views

Better default prior for non-negative canonical polyadic decomposition of counts than Exp(1)?

Suppose I have a instance of a random $k$-mode tensor $X_{n_1 \times \ldots \times n_k}$ of count data. I would like to perform non-negative canonical polyadic decomposition of this tensor using ...
Galen's user avatar
  • 9,381
1 vote
1 answer
86 views

What "Convolution filters along the time axis" means?

Suppose that I have a tensor of height:25 and width:50. Height is my temporal axis, therefore I have a window of 25 time steps. Therefore my input tensor is: I want to extract temporal features / ...
Mas A's user avatar
  • 223
0 votes
1 answer
367 views

bos_token for a custom Transformer

I am trying to use a Transformer to solve a time-series problem. I built the model using the Pytorch library. And I am planning to train the model from scratch. The model is looking back last L time-...
Wenuka's user avatar
  • 113
4 votes
1 answer
64 views

Einstein notation $-$ or another $-$ to denote constraints in high dimensional ILP problems

When discussing marginal sums of arrays in 3 dimensions or more, is it customary in the statistical and/or data science communities to use the Einstein summation convention? Is some other form ...
Peter Leopold's user avatar
1 vote
2 answers
87 views

What does Cayley's hyperdeterminant of a 2x2x2 mixed-product moment tensor tell us about how two variables are related?

Suppose we have a collection of random variables $S = \{ X_0, X_1 \}$ encoded into the $2 \times 2 \times 2$ tensor $$\mathcal{C}[i, j, k] = \mathbb{E}[X_i X_j X_k]$$ where $X_i, X_j, X_k \in S$ and $...
Galen's user avatar
  • 9,381
1 vote
0 answers
65 views

Is there a multilinear kernel principal components analysis?

PCA can be extended to kPCA using the kernel trick. MPCA is a multilinear extension of PCA that involves multiple matrices for the different modes of the data tensor. Can MCPA be similarly extended ...
Galen's user avatar
  • 9,381
1 vote
0 answers
34 views

Is there a multilinear principal component regression?

PCR is a linear regression problem on top of PCA. Analogically, is there a 'multilinear PCR' as a linear or multilinear regression problem (e.g. CP tensor regression and Tucker tensor regression for ...
Galen's user avatar
  • 9,381
3 votes
2 answers
354 views

What is multilinear principal components analysis?

I've gotten a lot of usage out of principal component analysis, and after recently learning the basics of performing canonical polyadic decomposition I was intrigued to learn that there exists a ...
Galen's user avatar
  • 9,381
2 votes
0 answers
37 views

Is there an improved canonical polyadic decomposition for symmetric tensors?

Let us suppose I want to find a CP decomposition of a $n$-mode tensor $\mathcal{A}$. Fortunately the tensor has the permutation symmetry $$\mathcal{A}[i_1, \cdots, i_n] = \mathcal{A}[\sigma (i_1), \...
Galen's user avatar
  • 9,381
1 vote
0 answers
106 views

How to select ranks to search for CP decomposition of 4-mode tensor of experimental gene expression data?

I have a sparse (0.625 percent non-zero occupancy) data tensor with shape (118, 16, 5009, 10). I would like to try exploring the data using CP decomposition. The ...
Galen's user avatar
  • 9,381
1 vote
1 answer
231 views

Presenting 2D smoother of GAM

I had a look around and couldn't find the answer to my problem, so hopefully this is a new question. I tried to fit a GAM with two continuous explanatory variables, one of them is Day of the year and ...
Toastershock91's user avatar
1 vote
0 answers
16 views

CP Tensor Decomposition and Correlating Sample Magnitudes with Variables of Interest

I am learning about tensor decomposition, specifically CP, and am trying to understand if I can use it for my research. To give a bit more detail, I have brain imaging data from 10 participants, with ...
pdhami's user avatar
  • 335
2 votes
1 answer
1k views

Mathematical representation of 1D convolution

How does one write the mathematical formula for conv1d used in PyTorch, including parameters like stride length and padding? For instance, I can write ...
Kevin's user avatar
  • 73
1 vote
1 answer
148 views

How to express the notion of a vector, where every element inside is matrix?

I met a question when I tried to express a vector, where every element inside is a matrix. We know the notion usually works like this: scalar: $a$ vector: $\boldsymbol{a}$ matrix: $\boldsymbol A$ ...
Nick Nick Nick's user avatar
1 vote
1 answer
385 views

R/mgcv: How do you interpret the 'fixed effects' for multivariate te() tensor products in the 'lme' part of a gamm model in R?

When fitting a 'gamm' model in the R package mgcv, and using a te() tensor product of three variables, the lme part of the model reports seven fixed effects, from '...
Alvaro Rucua's user avatar
1 vote
0 answers
20 views

Are the arrows connecting nodes called as Tensors?

I asked a close friend of mine to explain Neural Networks at high level. At one point, he explained to me the arrows that connecting nodes are called as Tensors and that each tensor has a weight. I ...
jeffbRTC's user avatar
  • 111
0 votes
1 answer
1k views

Normalized 2D tensor values are not in range 0-1

Below function takes in 2D tensor and normalizes it using broadcasting .The issue is except all values to be in range 0-1 but the result has values outside this range . How to get all values in 2D ...
star's user avatar
  • 135
1 vote
0 answers
605 views

Tensor linearization interview question [closed]

I got the following question in a coding interview for machine learning engineer position. Write a function: ...
Encipher's user avatar
  • 175
1 vote
0 answers
24 views

What are the basic statistics to describe tensor data distribution [duplicate]

For numeric data, the basic/straightforward way to describe its distribution is to use some metrics like mean, min, max, std et al. What about tensor data (vectors and high dimensional data)? Of ...
cxs1031's user avatar
  • 31
1 vote
1 answer
656 views

Reverse-Mode Automatic Differentiation with respect to a Matrix: How to "Matrix Multiply" 4D Tensors?

This is a follow up question I have on this excellent answer: https://stats.stackexchange.com/a/235758/307400. I will save me writing down any details about reverse-mode automatic differentiation, the ...
cherrywoods's user avatar
2 votes
1 answer
617 views

How can we perform matrix factorisation for three dimensional matrix?

I am working on the recommendation system in which I have three factors user_id, time and ...
Lavanya Sagunthala's user avatar
5 votes
1 answer
110 views

obtaining 4th moment tensor under change of coordinates

Suppose I have a random real-valued vector $x=x_1,\ldots,x_d$ and $M_{ijkl}=E[x_i x_j x_k x_l]$ and apply a change of coordinates $y=Ax$ where $A$ is an orthogonal matrix. How do I obtain $N_{ijkl}=E[...
Yaroslav Bulatov's user avatar
1 vote
0 answers
103 views

How to find eigenvalues and eigenvectors of the cokurtosis matrix?

Kurtosis is the fourth statistical moment of a random variable's distribution. Unlike the variance-covariance matrix $\Sigma$, which had a shape of $p\times p$, the kurtosis-cokurtosis matrix is ...
develarist's user avatar
  • 4,015
2 votes
0 answers
58 views

Tensor product between an ispline and a bspline for fitting data that should be monotonic in one dimension

I'm not very familiar with the process for solving tensor product basis fittings. I've done some work with fitting an ispline basis with a non-negative-least-squares solver to fit a monotonic spline ...
Joey's user avatar
  • 21
1 vote
1 answer
84 views

What is the actual use of GAN model? Is it only used to generate the data that closely resembles original dataset?

I am very new to tensor flow. I came across GANS. From what I understand in GANs there are 2 models, Generator and Discriminator. Generator job is to generate the data that will be able to fool the ...
Sam's user avatar
  • 113
0 votes
0 answers
23 views

How to derive mathematically that derivative of |Ax-y|^2 with respect to A is 2|Ax-y| x^T [duplicate]

How to get transpose part when derive mathematically $$ \frac {\partial|Ax-y|^2}{\partial A} = 2|Ax-y|x^T $$
Rishi Jaiswal's user avatar
0 votes
2 answers
188 views

Understandable way of thinking about higher order tensors

I study deep learning and the one of the major problem I face is I can't imagine shape of higher order tensors in my head. for instance - A 2d tensor - (x,y) is a rectangle with x,y along its length ...
Beginner's user avatar
  • 103
1 vote
0 answers
395 views

Compact/Vectorized Multiclass Logistic Regression Hessian

I know that the Hessian of the categorical cross entropy w.r.t the weights is given by $$\frac{\partial^2 L}{\partial w^2} = \sum_{i=1}^{m} (Diag(\hat{y}_i)-\hat{y}_i^T \hat{y}_i) \otimes x_i^T x_i$$ ...
nubol23's user avatar
  • 33
11 votes
2 answers
2k views

"row" and "column" are the names of axes of 2d array, is there a similar naming for a 3d array?

row and column are the names of axes of 2d array. this python array, array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) could be viewed as a matrix that ...
whnlp's user avatar
  • 237
3 votes
0 answers
82 views

Why are some robust algorithms valid for Tucker decomposition, but not for CP decomposition?

I have been reading up about CP and Tucker decomposition. It makes sense that CP decomposition is a special case of Tucker decomposition, where the core tensor is super-diagonal. However, if this is ...
Roy's user avatar
  • 166
2 votes
0 answers
240 views

Diffusion tensor as a covariance matrix

TLDR: In nuclear magnetic resonance (NMR), to study molecular diffusion we assume that molecules displace in 3D space according to a trivariate gaussian distribution. The variables are then the ...
user241848's user avatar
4 votes
1 answer
749 views

Higher moments of linear regression residuals?

I previously asked this on Math StackExchange, with no success, but this post will add to that with some simulations. Background In the following linear regression with i.i.d $\epsilon_i$ $(i = 1, \...
Tom Chen's user avatar
  • 621
2 votes
0 answers
361 views

Regarding the quantics tensor train (QTT) format

I originally posted this question in Data Science Stack Exchange, however, I think this forum may be better for this question. I believe I have a fair understanding of the tensor train (TT) format, ...
grad_student's user avatar
0 votes
0 answers
528 views

Random forest for tensors

Say we have have input tensor $X \in \mathbb{R}^{T \times N \times P}$ and output tensor $Y \in \mathbb{R}^{T \times N \times K}$, and we aim to build a Random Forests model $Y = f(X)+ \epsilon$. ...
adam's user avatar
  • 115
2 votes
1 answer
2k views

Can a Fully Connected layer transform a 4D tensor to a 3D tensor by itself?

Recently, I was researching some topics in biometrics and I stumbled upon this paper. They have a table there (Table 1) in which they state that they used a modified CNN from this paper (Table 9). In ...
Colonder's user avatar
  • 125
3 votes
1 answer
8k views

Machine learning methods for multi-dimensional input and output

I have a large dataset where my input is an $M$-dimensional tensor, and each input has a corresponding $N$-dimensional output. My goal is to train a method to learn outputs from the millions of inputs ...
Mathews24's user avatar
  • 589
2 votes
0 answers
82 views

Is there any sort of quadratic SVD for dimensionality reduction?

X-Posted on math.stackexchange, apologies, though I thought this was equally relevant to both communities. I'm wondering if there exists any higher-order SVD for dimensionality reduction. Note that ...
user650261's user avatar
0 votes
1 answer
129 views

Parameters in a neural tensor network

I am reading the paper of "Reasoning With Neural Tensor Networks for Knowledge Base Completion". I read it many times but I couldn't understand the parameters that are used especially the parameter U. ...
bttX's user avatar
  • 3
0 votes
1 answer
524 views

Best way to represent 3D data for Neural Networks

I want to train a generative model over a dataset where each example is a $X = (N,3)$ matrix representing $N$ points in $\mathbb{R}^3$. The local structure (i.e. the correlations between neighboring ...
Toool's user avatar
  • 155
1 vote
1 answer
199 views

Satellite data pre-processing for Keras CNNs [closed]

I’m looking at satellite data and want to do object detection using CNNs in Keras. I’m currently pre-processing the data (turning them into tensors) that I’ve obtained which include the original ...
FinanceGardener's user avatar
0 votes
1 answer
224 views

Shape of a tensor [closed]

Suppose I have a variable that looks like this [[[1., 2., 3.]], [[7., 8., 9.]]] I have read this is a rank 3 tensor with shape ...
iratelilkid's user avatar