Questions tagged [tensor]
In machine learning, tensor is a multidimensional (multi-index, or multi-way) array of numbers, i.e. a generalization of a matrix.
53
questions
1
vote
1
answer
15
views
Presenting 2D smoother of GAM
I had a look around and couldn't find the answer to my problem, so hopefully this is a new question.
I tried to fit a GAM with two continuous explanatory variables, one of them is Day of the year and ...
1
vote
0
answers
19
views
Comparing AIC of Tensor Product Smooths versus Thin Plate Splines
I'm comparing the AIC of these two models.
Tensor Product Smooth vs. Thin Plate Spline both fit using REML
...
- 542
0
votes
0
answers
14
views
Model predicts same number for any input on initialization of random weights
My PNAConv (pytorch) network has been having issues with predicting the same exact value for all inputs. Without getting into too many details, I have a broader neural netowrk question.
When I ask my ...
0
votes
0
answers
6
views
CP Tensor Decomposition and Correlating Sample Magnitudes with Variables of Interest
I am learning about tensor decomposition, specifically CP, and am trying to understand if I can use it for my research.
To give a bit more detail, I have brain imaging data from 10 participants, with ...
- 295
2
votes
1
answer
215
views
Mathematical representation of 1D convolution
How does one write the mathematical formula for conv1d used in PyTorch, including parameters like stride length and padding?
For instance, I can write
...
- 73
0
votes
0
answers
11
views
Higher order tensors for describe (hyper) graphs and features attached
I'm trying to understand better some notation I'm reading in the following paper:
https://arxiv.org/pdf/1901.09342.pdf
In particular, graph or hyper graph data can be described by using tensors $\...
- 437
0
votes
0
answers
10
views
Differentiating a function with respect to a matrix [duplicate]
I'm new to matrix calculus and I'm trying to find the formulas for matrix differentiation.
e.g.
$\frac{\partial f}{\partial z}$ = zzx
where z is a KxK matrix, and x is a vector in K
I found a few ...
- 101
1
vote
1
answer
47
views
How to express the notion of a vector, where every element inside is matrix?
I met a question when I tried to express a vector, where every element inside is a matrix. We know the notion usually works like this:
scalar: $a$
vector: $\boldsymbol{a}$
matrix: $\boldsymbol A$
...
- 115
1
vote
1
answer
160
views
R/mgcv: How do you interpret the 'fixed effects' for multivariate te() tensor products in the 'lme' part of a gamm model in R?
When fitting a 'gamm' model in the R package mgcv, and using a te() tensor product of three variables, the lme part of the model reports seven fixed effects, from '...
- 13
1
vote
0
answers
17
views
Are the arrows connecting nodes called as Tensors?
I asked a close friend of mine to explain Neural Networks at high level. At one point, he explained to me the arrows that connecting nodes are called as Tensors and that each tensor has a weight.
I ...
- 111
0
votes
1
answer
659
views
Normalized 2D tensor values are not in range 0-1
Below function takes in 2D tensor and normalizes it using broadcasting .The issue is except all values to be in range 0-1 but the result has values outside this range . How to get all values in 2D ...
- 135
1
vote
0
answers
225
views
Tensor linearization interview question [closed]
I got the following question in a coding interview for machine learning engineer position.
Write a function:
...
- 121
1
vote
0
answers
22
views
What are the basic statistics to describe tensor data distribution [duplicate]
For numeric data, the basic/straightforward way to describe its distribution is to use some metrics like mean, min, max, std et al.
What about tensor data (vectors and high dimensional data)?
Of ...
- 31
1
vote
0
answers
144
views
Reverse-Mode Automatic Differentiation with respect to a Matrix: How to "Matrix Multiply" 4D Tensors?
This is a follow up question I have on this excellent answer: https://stats.stackexchange.com/a/235758/307400. I will save me writing down any details about reverse-mode automatic differentiation, the ...
- 111
2
votes
1
answer
178
views
How can we perform matrix factorisation for three dimensional matrix?
I am working on the recommendation system in which I have three factors user_id, time and ...
5
votes
1
answer
77
views
obtaining 4th moment tensor under change of coordinates
Suppose I have a random real-valued vector $x=x_1,\ldots,x_d$ and $M_{ijkl}=E[x_i x_j x_k x_l]$ and apply a change of coordinates $y=Ax$ where $A$ is an orthogonal matrix. How do I obtain $N_{ijkl}=E[...
- 5,291
1
vote
0
answers
56
views
How to find eigenvalues and eigenvectors of the cokurtosis matrix?
Kurtosis is the fourth statistical moment of a random variable's distribution. Unlike the variance-covariance matrix $\Sigma$, which had a shape of $p\times p$, the kurtosis-cokurtosis matrix is ...
- 3,089
2
votes
0
answers
37
views
Tensor product between an ispline and a bspline for fitting data that should be monotonic in one dimension
I'm not very familiar with the process for solving tensor product basis fittings. I've done some work with fitting an ispline basis with a non-negative-least-squares solver to fit a monotonic spline ...
- 21
1
vote
1
answer
58
views
What is the actual use of GAN model? Is it only used to generate the data that closely resembles original dataset?
I am very new to tensor flow. I came across GANS. From what I understand in GANs there are 2 models, Generator and Discriminator. Generator job is to generate the data that will be able to fool the ...
- 113
0
votes
0
answers
22
views
How to derive mathematically that derivative of |Ax-y|^2 with respect to A is 2|Ax-y| x^T [duplicate]
How to get transpose part when derive mathematically $$ \frac {\partial|Ax-y|^2}{\partial A} = 2|Ax-y|x^T $$
0
votes
2
answers
92
views
Understandable way of thinking about higher order tensors
I study deep learning and the one of the major problem I face is I can't imagine shape of higher order tensors in my head.
for instance -
A 2d tensor - (x,y) is a rectangle with x,y along its length ...
- 103
1
vote
0
answers
330
views
Compact/Vectorized Multiclass Logistic Regression Hessian
I know that the Hessian of the categorical cross entropy w.r.t the weights is given by
$$\frac{\partial^2 L}{\partial w^2} = \sum_{i=1}^{m} (Diag(\hat{y}_i)-\hat{y}_i^T \hat{y}_i) \otimes x_i^T x_i$$
...
- 33
5
votes
1
answer
291
views
"row" and "column" are the names of axes of 2d array, is there a similar naming for a 3d array?
row and column are the names of axes of 2d array.
this python array,
array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
could be viewed as a matrix that ...
- 157
1
vote
0
answers
57
views
Why are some robust algorithms valid for Tucker decomposition, but not for CP decomposition?
I have been reading up about CP and Tucker decomposition.
It makes sense that CP decomposition is a special case of Tucker decomposition, where the core tensor is super-diagonal. However, if this is ...
- 111
1
vote
0
answers
138
views
Diffusion tensor as a covariance matrix
TLDR: In nuclear magnetic resonance (NMR), to study molecular diffusion we assume that molecules displace in 3D space according to a trivariate gaussian distribution. The variables are then the ...
- 11
3
votes
1
answer
448
views
Higher moments of linear regression residuals?
I previously asked this on Math StackExchange, with no success, but this post will add to that with some simulations.
Background
In the following linear regression with i.i.d $\epsilon_i$ $(i = 1, \...
- 471
2
votes
0
answers
200
views
Regarding the quantics tensor train (QTT) format
I originally posted this question in Data Science Stack Exchange, however, I think this forum may be better for this question.
I believe I have a fair understanding of the tensor train (TT) format, ...
- 21
0
votes
0
answers
426
views
Random forest for tensors
Say we have have input tensor $X \in \mathbb{R}^{T \times N \times P}$ and output tensor $Y \in \mathbb{R}^{T \times N \times K}$, and we aim to build a Random Forests model $Y = f(X)+ \epsilon$.
...
- 95
2
votes
1
answer
2k
views
Can a Fully Connected layer transform a 4D tensor to a 3D tensor by itself?
Recently, I was researching some topics in biometrics and I stumbled upon this paper. They have a table there (Table 1) in which they state that they used a modified CNN from this paper (Table 9). In ...
- 125
2
votes
1
answer
6k
views
Machine learning methods for multi-dimensional input and output
I have a large dataset where my input is an $M$-dimensional tensor, and each input has a corresponding $N$-dimensional output. My goal is to train a method to learn outputs from the millions of inputs ...
- 447
2
votes
0
answers
71
views
Is there any sort of higher-order SVD (quadratic and above) for dimensionality reduction?
X-Posted on math.stackexchange, apologies, though I thought this was equally relevant to both communities.
I'm wondering if there exists any higher-order SVD for dimensionality reduction. Note that ...
- 121
0
votes
1
answer
87
views
Parameters in a neural tensor network
I am reading the paper of "Reasoning With Neural Tensor Networks for Knowledge Base Completion". I read it many times but I couldn't understand the parameters that are used especially the parameter U. ...
- 3
0
votes
1
answer
437
views
Best way to represent 3D data for Neural Networks
I want to train a generative model over a dataset where each example is a $X = (N,3)$ matrix representing $N$ points in $\mathbb{R}^3$.
The local structure (i.e. the correlations between neighboring ...
- 145
1
vote
1
answer
181
views
Satellite data pre-processing for Keras CNNs [closed]
I’m looking at satellite data and want to do object detection using CNNs in Keras. I’m currently pre-processing the data (turning them into tensors) that I’ve obtained which include the original ...
0
votes
1
answer
152
views
Shape of a tensor [closed]
Suppose I have a variable that looks like this
[[[1., 2., 3.]], [[7., 8., 9.]]]
I have read this is a rank 3 tensor with shape ...
- 91
0
votes
0
answers
83
views
References on tensor algebra for machine learning? [duplicate]
So tensors come up a lot in machine learning in various settings. Any math major will have studied linear algebra heavily and it's usually much simpler to use linear operators and the corresponding ...
- 1
2
votes
3
answers
324
views
Tensor Classification Models
Aside from Convolution Neural Networks, are there any other methods that allow for classification of Tensors? My observations consist of multi-dimensional tensors with height of 1, where each channel ...
- 181
1
vote
1
answer
1k
views
CP decomposition for tensor factorization
I am trying to understand CP decomposition for a three way tensor. Lets have a tensor which has the dimensions I by J by K. When we apply CP decomposition, it decomposes the tensor as a sum of a rank-...
- 95
0
votes
1
answer
437
views
Fitting Tensor Product P-splines, Penalty Parameters
I am working with the mgcv package in r and I am fitting tensor product P-splines.
...
- 33
1
vote
2
answers
629
views
Implementation of algorithm to determine next basket recommendation?
I hope I am asking in right forum, and forgive me, if I am wrong.
I am trying to implement an algorithm based on an open paper ...
- 133
0
votes
0
answers
32
views
Validating decomposition of Synthetic Tensor generated from Unevenly Sampled Tensor
I have a 3-way tensor generated from 7 experiments, with each experiment being matricized and becoming a frontal slice of the tensor (thus mode-3 is of length 7). The data is generated from ...
28
votes
3
answers
45k
views
Difference between samples, time steps and features in neural network
I am going through the following blog on LSTM neural network:
http://machinelearningmastery.com/understanding-stateful-lstm-recurrent-neural-networks-python-keras/
The author reshapes the input ...
- 383
1
vote
1
answer
59
views
Tensor method under high noise
Tensor methods give good results under low approximation error. (e.g. at 7min http://videolectures.net/iclr2016_anandkumar_nonconvex_learning/). I am wondering how do they do when the noise in the ...
- 151
6
votes
1
answer
1k
views
Usage of tensor notation in statistics
A friend of mine (mathematician) basically told me I shouldn't bother with matrix algebra and should focus on tensor analysis/manipulation. He said it's much more general and intuitive. I've been ...
- 71
19
votes
2
answers
2k
views
Tensors in neural network literature: what's the simplest definition out there?
In the neural network literature, often we encounter the word "tensor".
Is it different from a vector? And from a matrix? Have you got any specific example that clarifies its definition?
I'm a bit ...
- 712
1
vote
0
answers
265
views
Logistic Tensor Regression
Say there are users and they view articles and click (or not click) on articles. I represent the $i$-th user as $x_i$, a $D \times1 $ vector and $j$-th article as $z_j$, a $C \times 1$vector. The ...
- 123
1
vote
1
answer
265
views
Is Tensorflow shape just for convenience? [closed]
It's pretty neat that Tensorflow allows you to define and do math on arbitrary tensors, but for supervised learning applications, is there any reason you would want to define your inputs and outputs ...
- 638
5
votes
0
answers
222
views
Intuition behind using tensors ? [closed]
I am trying to build a recommendation model by using a tensor. In order to recommend an article to a user, I have built a model to predict users' preferable articles.
I am labeling articles based on ...
- 1,117
191
votes
10
answers
43k
views
Why the sudden fascination with tensors?
I've noticed lately that a lot of people are developing tensor equivalents of many methods (tensor factorization, tensor kernels, tensors for topic modeling, etc) I'm wondering, why is the world ...
7
votes
1
answer
1k
views
The meaning of tensors in the neural network community [duplicate]
In the neural network community, is a tensor pretty much always just a multi-dimensional array?
