Questions tagged [tensor]

In machine learning, tensor is a multidimensional (multi-index, or multi-way) array of numbers, i.e. a generalization of a matrix.

Filter by
Sorted by
Tagged with
1 vote
1 answer
15 views

Presenting 2D smoother of GAM

I had a look around and couldn't find the answer to my problem, so hopefully this is a new question. I tried to fit a GAM with two continuous explanatory variables, one of them is Day of the year and ...
user avatar
1 vote
0 answers
19 views

Comparing AIC of Tensor Product Smooths versus Thin Plate Splines

I'm comparing the AIC of these two models. Tensor Product Smooth vs. Thin Plate Spline both fit using REML ...
user avatar
  • 542
0 votes
0 answers
14 views

Model predicts same number for any input on initialization of random weights

My PNAConv (pytorch) network has been having issues with predicting the same exact value for all inputs. Without getting into too many details, I have a broader neural netowrk question. When I ask my ...
user avatar
0 votes
0 answers
6 views

CP Tensor Decomposition and Correlating Sample Magnitudes with Variables of Interest

I am learning about tensor decomposition, specifically CP, and am trying to understand if I can use it for my research. To give a bit more detail, I have brain imaging data from 10 participants, with ...
user avatar
  • 295
2 votes
1 answer
215 views

Mathematical representation of 1D convolution

How does one write the mathematical formula for conv1d used in PyTorch, including parameters like stride length and padding? For instance, I can write ...
user avatar
  • 73
0 votes
0 answers
11 views

Higher order tensors for describe (hyper) graphs and features attached

I'm trying to understand better some notation I'm reading in the following paper: https://arxiv.org/pdf/1901.09342.pdf In particular, graph or hyper graph data can be described by using tensors $\...
user avatar
0 votes
0 answers
10 views

Differentiating a function with respect to a matrix [duplicate]

I'm new to matrix calculus and I'm trying to find the formulas for matrix differentiation. e.g. $\frac{\partial f}{\partial z}$ = zzx where z is a KxK matrix, and x is a vector in K I found a few ...
user avatar
1 vote
1 answer
47 views

How to express the notion of a vector, where every element inside is matrix?

I met a question when I tried to express a vector, where every element inside is a matrix. We know the notion usually works like this: scalar: $a$ vector: $\boldsymbol{a}$ matrix: $\boldsymbol A$ ...
user avatar
1 vote
1 answer
160 views

R/mgcv: How do you interpret the 'fixed effects' for multivariate te() tensor products in the 'lme' part of a gamm model in R?

When fitting a 'gamm' model in the R package mgcv, and using a te() tensor product of three variables, the lme part of the model reports seven fixed effects, from '...
user avatar
1 vote
0 answers
17 views

Are the arrows connecting nodes called as Tensors?

I asked a close friend of mine to explain Neural Networks at high level. At one point, he explained to me the arrows that connecting nodes are called as Tensors and that each tensor has a weight. I ...
user avatar
  • 111
0 votes
1 answer
659 views

Normalized 2D tensor values are not in range 0-1

Below function takes in 2D tensor and normalizes it using broadcasting .The issue is except all values to be in range 0-1 but the result has values outside this range . How to get all values in 2D ...
user avatar
  • 135
1 vote
0 answers
225 views

Tensor linearization interview question [closed]

I got the following question in a coding interview for machine learning engineer position. Write a function: ...
user avatar
  • 121
1 vote
0 answers
22 views

What are the basic statistics to describe tensor data distribution [duplicate]

For numeric data, the basic/straightforward way to describe its distribution is to use some metrics like mean, min, max, std et al. What about tensor data (vectors and high dimensional data)? Of ...
user avatar
  • 31
1 vote
0 answers
144 views

Reverse-Mode Automatic Differentiation with respect to a Matrix: How to "Matrix Multiply" 4D Tensors?

This is a follow up question I have on this excellent answer: https://stats.stackexchange.com/a/235758/307400. I will save me writing down any details about reverse-mode automatic differentiation, the ...
user avatar
2 votes
1 answer
178 views

How can we perform matrix factorisation for three dimensional matrix?

I am working on the recommendation system in which I have three factors user_id, time and ...
user avatar
5 votes
1 answer
77 views

obtaining 4th moment tensor under change of coordinates

Suppose I have a random real-valued vector $x=x_1,\ldots,x_d$ and $M_{ijkl}=E[x_i x_j x_k x_l]$ and apply a change of coordinates $y=Ax$ where $A$ is an orthogonal matrix. How do I obtain $N_{ijkl}=E[...
user avatar
1 vote
0 answers
56 views

How to find eigenvalues and eigenvectors of the cokurtosis matrix?

Kurtosis is the fourth statistical moment of a random variable's distribution. Unlike the variance-covariance matrix $\Sigma$, which had a shape of $p\times p$, the kurtosis-cokurtosis matrix is ...
user avatar
  • 3,089
2 votes
0 answers
37 views

Tensor product between an ispline and a bspline for fitting data that should be monotonic in one dimension

I'm not very familiar with the process for solving tensor product basis fittings. I've done some work with fitting an ispline basis with a non-negative-least-squares solver to fit a monotonic spline ...
user avatar
  • 21
1 vote
1 answer
58 views

What is the actual use of GAN model? Is it only used to generate the data that closely resembles original dataset?

I am very new to tensor flow. I came across GANS. From what I understand in GANs there are 2 models, Generator and Discriminator. Generator job is to generate the data that will be able to fool the ...
user avatar
  • 113
0 votes
0 answers
22 views

How to derive mathematically that derivative of |Ax-y|^2 with respect to A is 2|Ax-y| x^T [duplicate]

How to get transpose part when derive mathematically $$ \frac {\partial|Ax-y|^2}{\partial A} = 2|Ax-y|x^T $$
user avatar
0 votes
2 answers
92 views

Understandable way of thinking about higher order tensors

I study deep learning and the one of the major problem I face is I can't imagine shape of higher order tensors in my head. for instance - A 2d tensor - (x,y) is a rectangle with x,y along its length ...
user avatar
  • 103
1 vote
0 answers
330 views

Compact/Vectorized Multiclass Logistic Regression Hessian

I know that the Hessian of the categorical cross entropy w.r.t the weights is given by $$\frac{\partial^2 L}{\partial w^2} = \sum_{i=1}^{m} (Diag(\hat{y}_i)-\hat{y}_i^T \hat{y}_i) \otimes x_i^T x_i$$ ...
user avatar
  • 33
5 votes
1 answer
291 views

"row" and "column" are the names of axes of 2d array, is there a similar naming for a 3d array?

row and column are the names of axes of 2d array. this python array, array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) could be viewed as a matrix that ...
user avatar
  • 157
1 vote
0 answers
57 views

Why are some robust algorithms valid for Tucker decomposition, but not for CP decomposition?

I have been reading up about CP and Tucker decomposition. It makes sense that CP decomposition is a special case of Tucker decomposition, where the core tensor is super-diagonal. However, if this is ...
user avatar
  • 111
1 vote
0 answers
138 views

Diffusion tensor as a covariance matrix

TLDR: In nuclear magnetic resonance (NMR), to study molecular diffusion we assume that molecules displace in 3D space according to a trivariate gaussian distribution. The variables are then the ...
user avatar
3 votes
1 answer
448 views

Higher moments of linear regression residuals?

I previously asked this on Math StackExchange, with no success, but this post will add to that with some simulations. Background In the following linear regression with i.i.d $\epsilon_i$ $(i = 1, \...
user avatar
  • 471
2 votes
0 answers
200 views

Regarding the quantics tensor train (QTT) format

I originally posted this question in Data Science Stack Exchange, however, I think this forum may be better for this question. I believe I have a fair understanding of the tensor train (TT) format, ...
user avatar
0 votes
0 answers
426 views

Random forest for tensors

Say we have have input tensor $X \in \mathbb{R}^{T \times N \times P}$ and output tensor $Y \in \mathbb{R}^{T \times N \times K}$, and we aim to build a Random Forests model $Y = f(X)+ \epsilon$. ...
user avatar
  • 95
2 votes
1 answer
2k views

Can a Fully Connected layer transform a 4D tensor to a 3D tensor by itself?

Recently, I was researching some topics in biometrics and I stumbled upon this paper. They have a table there (Table 1) in which they state that they used a modified CNN from this paper (Table 9). In ...
user avatar
  • 125
2 votes
1 answer
6k views

Machine learning methods for multi-dimensional input and output

I have a large dataset where my input is an $M$-dimensional tensor, and each input has a corresponding $N$-dimensional output. My goal is to train a method to learn outputs from the millions of inputs ...
user avatar
  • 447
2 votes
0 answers
71 views

Is there any sort of higher-order SVD (quadratic and above) for dimensionality reduction?

X-Posted on math.stackexchange, apologies, though I thought this was equally relevant to both communities. I'm wondering if there exists any higher-order SVD for dimensionality reduction. Note that ...
user avatar
0 votes
1 answer
87 views

Parameters in a neural tensor network

I am reading the paper of "Reasoning With Neural Tensor Networks for Knowledge Base Completion". I read it many times but I couldn't understand the parameters that are used especially the parameter U. ...
user avatar
  • 3
0 votes
1 answer
437 views

Best way to represent 3D data for Neural Networks

I want to train a generative model over a dataset where each example is a $X = (N,3)$ matrix representing $N$ points in $\mathbb{R}^3$. The local structure (i.e. the correlations between neighboring ...
user avatar
  • 145
1 vote
1 answer
181 views

Satellite data pre-processing for Keras CNNs [closed]

I’m looking at satellite data and want to do object detection using CNNs in Keras. I’m currently pre-processing the data (turning them into tensors) that I’ve obtained which include the original ...
user avatar
0 votes
1 answer
152 views

Shape of a tensor [closed]

Suppose I have a variable that looks like this [[[1., 2., 3.]], [[7., 8., 9.]]] I have read this is a rank 3 tensor with shape ...
user avatar
0 votes
0 answers
83 views

References on tensor algebra for machine learning? [duplicate]

So tensors come up a lot in machine learning in various settings. Any math major will have studied linear algebra heavily and it's usually much simpler to use linear operators and the corresponding ...
user avatar
  • 1
2 votes
3 answers
324 views

Tensor Classification Models

Aside from Convolution Neural Networks, are there any other methods that allow for classification of Tensors? My observations consist of multi-dimensional tensors with height of 1, where each channel ...
user avatar
  • 181
1 vote
1 answer
1k views

CP decomposition for tensor factorization

I am trying to understand CP decomposition for a three way tensor. Lets have a tensor which has the dimensions I by J by K. When we apply CP decomposition, it decomposes the tensor as a sum of a rank-...
user avatar
0 votes
1 answer
437 views

Fitting Tensor Product P-splines, Penalty Parameters

I am working with the mgcv package in r and I am fitting tensor product P-splines. ...
user avatar
  • 33
1 vote
2 answers
629 views

Implementation of algorithm to determine next basket recommendation?

I hope I am asking in right forum, and forgive me, if I am wrong. I am trying to implement an algorithm based on an open paper ...
user avatar
  • 133
0 votes
0 answers
32 views

Validating decomposition of Synthetic Tensor generated from Unevenly Sampled Tensor

I have a 3-way tensor generated from 7 experiments, with each experiment being matricized and becoming a frontal slice of the tensor (thus mode-3 is of length 7). The data is generated from ...
user avatar
28 votes
3 answers
45k views

Difference between samples, time steps and features in neural network

I am going through the following blog on LSTM neural network: http://machinelearningmastery.com/understanding-stateful-lstm-recurrent-neural-networks-python-keras/ The author reshapes the input ...
user avatar
1 vote
1 answer
59 views

Tensor method under high noise

Tensor methods give good results under low approximation error. (e.g. at 7min http://videolectures.net/iclr2016_anandkumar_nonconvex_learning/). I am wondering how do they do when the noise in the ...
user avatar
  • 151
6 votes
1 answer
1k views

Usage of tensor notation in statistics

A friend of mine (mathematician) basically told me I shouldn't bother with matrix algebra and should focus on tensor analysis/manipulation. He said it's much more general and intuitive. I've been ...
user avatar
19 votes
2 answers
2k views

Tensors in neural network literature: what's the simplest definition out there?

In the neural network literature, often we encounter the word "tensor". Is it different from a vector? And from a matrix? Have you got any specific example that clarifies its definition? I'm a bit ...
user avatar
1 vote
0 answers
265 views

Logistic Tensor Regression

Say there are users and they view articles and click (or not click) on articles. I represent the $i$-th user as $x_i$, a $D \times1 $ vector and $j$-th article as $z_j$, a $C \times 1$vector. The ...
user avatar
  • 123
1 vote
1 answer
265 views

Is Tensorflow shape just for convenience? [closed]

It's pretty neat that Tensorflow allows you to define and do math on arbitrary tensors, but for supervised learning applications, is there any reason you would want to define your inputs and outputs ...
user avatar
  • 638
5 votes
0 answers
222 views

Intuition behind using tensors ? [closed]

I am trying to build a recommendation model by using a tensor. In order to recommend an article to a user, I have built a model to predict users' preferable articles. I am labeling articles based on ...
user avatar
  • 1,117
191 votes
10 answers
43k views

Why the sudden fascination with tensors?

I've noticed lately that a lot of people are developing tensor equivalents of many methods (tensor factorization, tensor kernels, tensors for topic modeling, etc) I'm wondering, why is the world ...
7 votes
1 answer
1k views

The meaning of tensors in the neural network community [duplicate]

In the neural network community, is a tensor pretty much always just a multi-dimensional array?
user avatar