Questions tagged [embeddings]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
3 views

What is gravity in the context of “folding” in recommender systems?

What is "gravity" in the context of recommender systems? More specifically, how is it supposed to help with the "folding" problem where irrelevant queries may be returned if we don't provide ...
0
votes
1answer
25 views

Neural network backpropagation to update inputs, not weights (e.g. fine-tuning embeddings)?

I recently re-read Stanford CS231N lecture notes on computer vision and backpropagation, and I came across this passage (emphasis mine): Note that (as is usually the case in Machine Learning) we ...
1
vote
0answers
13 views

Adam converges while SGD does not improve at all

I am trying to build a model based movie recommendation system with a neural network. The architecture looks as follows: ...
0
votes
0answers
18 views

Calculating similarities between two populations using embeddings

I would like to find items from population B that are most similar to an item from population A. I have the following set up: Two sparse datasets where each row is an item (treat row index as item ID)...
2
votes
0answers
47 views

Why BERT use learned positional embedding?

Compared with sinusoidal positional encoding used in Transformer, BERT's learned-lookup-table solution has 2 drawbacks in my mind: Fixed length Cannot reflect relative distance Could anyone please ...
0
votes
0answers
8 views

Integrate popularity with the approximate nearest neighbor searching?

I studied the mechanism of some ANN algorithms but only find that each stored vector is treated equally. That is, the popularity of the corresponding vectors are ignored. How can all vector ...
1
vote
1answer
49 views

What are state of the art methods for creating embeddings for sets?

I want to create embeddings in $R^D$ for sets. So I want a function (probably a neural network) that takes in a set $ S = \{ s_1, \dots, s_n \} $ (and ideally of any size, so the number of elements ...
0
votes
0answers
26 views

What's the best way to combine embeddings for ID list features?

I am using an embedding table to incorporate a high cardinality categorical feature into a model. The tricky part is that for 1 training observation this feature may have multiple values. For example, ...
0
votes
0answers
82 views

Embedding layer before LSTM layer

I am re-creating a clustering and churn prediction framework, cluschurn, which they deployed in production at Snap, Inc. In their research paper, paper_link, they use 14 days of user data and treat it ...
0
votes
0answers
27 views

Categorical Variables Encoding of Sets

I'm facing a problem in which I need to encode a categorical variable, which can take several values at the same time (and is basically a set), as input for a classifier. For instance, assuming the ...
0
votes
0answers
11 views

Is optimizing an embedding a convex or non-convex process?

Suppose we have input data with several thousand one-hot dimensions per element, representing, say, words in a passage of text. An embedding layer is a common feature at the top of machine learning ...
1
vote
3answers
49 views

General mathematical definition of a score

I understand what scores are in PCA, in particular this answer gives a good mathematical formulation: (Scores) are projections of the centred data in the linear space defined by the eigenvectors. ...
0
votes
1answer
46 views

Embedded markov chain example

I have an example of in my textbook of an "embeded markov chain", where I don't understand one step. Suppose that $(X_n)_{n\geq 0}$ is Markov$(\lambda, P)$. $\lambda$ is the initial distribution and ...
1
vote
1answer
28 views

Are the conditions of metric space satisfied in the latent space of a classification task?

Specifically, in the case of a neural network trained in a categorical classification task (cross-entropy loss function), does the final layer embedding space preserve the definition of distance ...
2
votes
0answers
23 views

What are embeddings in the context of machine learning?

I would like to find out an intuitive explanation of what are embeddings in the context of machine learning and neural networks. Is that essentially the same thing as a manifold? I've read a bunch of ...
0
votes
0answers
6 views

How to create linear user embedding from some answers to binary questions?

I have each user U_i answering 10 binary questions out of a pool Q with either answer 1 or 2. I would like to learn an embedding of user profiles based on these answers to predict is answer to other ...
0
votes
0answers
31 views

Using label encoder on a categorical feature that we want to embed

I have a dataset with feature that have very high cardinality, doing one-hot encoding is not an option because of memory limitations, so I am currently label encoding this feature and then I feed that ...
0
votes
0answers
40 views

t-sne embedding to medium-dimensions (e.g. 100 dimensions)?

I am using t-sne on 252 dimensional data to embed to lower-dimensions. I am curious to know if it is academically justifiable to embed it into medium dimensions such as 100 dimensions, or 80 ...
0
votes
0answers
11 views

Signal Embeddings using the skip-gram or CBOW model

So my work involves looking at a bunch of waveforms in the context of classifying events. I often am looking for new ways to represent my waveforms, and in my searching, I came across audio embeddings ...
1
vote
0answers
9 views

Neural network embedding layers allowing multiple class-membership features

Is there a version of embedding layers for neural networks that allows for multiple class-membership features? Any frameworks that have implemented this? E.g. imagine we are trying to predict ...
1
vote
0answers
38 views

BERT for non-textual sequence data

I'm working on a deep learning solution for classifying sequence data that isn't raw text but rather entities (which have already been extracted from the text). I am currently using word2vec-style ...
0
votes
0answers
27 views

how to use contextual embeddings?

I've read about pretrained word embeddings, and I understand how to use them. Basically, if I have the word nail (for example), there is a look up table where I can use the embedding for that word. ...
0
votes
1answer
13 views

Reconstructing face from randomised embedding

It is fairly agreed in literature that from a given face-embedding (that is a vector of features values) it is possible, with a good amount of effort, to reconstruct the original face, (See here for ...
0
votes
0answers
67 views

node2vec: Intuition behind BFS resulting in embeddings that capture structural equivalence

In the node2vec paper1 it is mentioned that when using BFS to embed nodes, the results correspond to structural equivalence (i.e. nodes that are "bridge nodes" would get embedded close together) ...
0
votes
0answers
39 views

Understanding method of time series delay embedding

As discussed e.g. in the paper by Muskulus and Verduyn-Lunel http://www.math.leidenuniv.nl/reports/files/2009-12.pdf the delay vector reconstruction of time series for a dynamical model allows to ...
2
votes
1answer
724 views

Is the Keras Embedding layer dependent on the target label?

I learned how to 'use' the Keras Embedding layer, but I am not able to find any more specific information about the actual behavior and training process of this layer. For now, I understand that the ...
2
votes
0answers
20 views

How to apply the diffusion maps when the matrix is PSD but not positivity preserving?

In order to apply the diffusion maps in a matrix $C\in\mathbb R^{n\times n}$ , that matrix must obey some restrictions, C is symmetric: $C_{ij} = C_{ji}$, C is positivity preserving (PP): $\forall ...
0
votes
1answer
80 views

Spectral embedding: interpretation of new dimensions

I'm trying to gain an intuition for the 2nd dimension in the spectral embedding of an S-shaped dataset as in this example: The 1st dimension seems to neatly capture the local similarity between ...
5
votes
1answer
403 views

What is the intuition behind the positional cosine encoding in the transformer network?

I don't understand how adding the cosine encodings/functions to each of the dimension of the word vector embedding enables the network to "understand" where each word is situated in the sentence. ...
2
votes
0answers
85 views

Can you use VAEs to produce deep word embeddings?

There are many articles about applications of VAE such as image reconstruction, denoising, data compression / augmentation. However, I have not seen an example of embeddings for high dimensional data ...
1
vote
0answers
40 views

Are training-loss optimised embeddings useless? (help resolve a disagreement)

The aim We are training a feed forward neural network as a regressor, with the aim of using the activations of the final layer as a type of embedding vector to represent the input examples. The ...
3
votes
1answer
425 views

Facebook's infersent intuition

When reviewing Infersent's architecture here, I noticed that, after encoding the premise and hypothesis to obtain two vectors u and v, they feed the set of fully connected layers with: (u, v) the ...
2
votes
1answer
27 views

Is there any theory on the order of Autoregression model for periodic time series? [closed]

Say M periodic signals, then one can safely say using AR-M model can achieve the perfect prediction. But how about further, in a more general sense, is there any publications on this? Update: Here ...
4
votes
0answers
297 views

Why researchers use conv1d for embeddings instead of dense layers?

In some papers (like Reinforcement learning for Vehicle Routing Problem), researchers use conv1d to embed the problem input into a hyperspace; for example, in solving TSP, they use conv1d on the (x,y) ...
2
votes
2answers
282 views

How to rank products using deep learning for recommender systems?

I am going to implement a recommender system based on this paper. It basically uses a double embedding technique, one for the user representation and another one for the products (movies, clothes, ...
1
vote
1answer
429 views

How do I add a missing word to a pretrained embedding?

I have a pretrained word embedding and want to add missing words to it. How exactly should I do that? I think to just randomly initialize the vector is not a good idea. I heard something about ...
4
votes
2answers
851 views

How to embed in Euclidean space

I have what I think might be a standard machine learning problem but I can't find a clear solution. I have lots vectors of different dimensions. For each pair of vectors I can compute their ...
2
votes
0answers
87 views

Recovering a distance matrix from nonnegative sparse correlation matrix?

After doing extensive literature research in all sorts of science I am completely puzzled. I am trying to find out what the state-of-the-art techniques would be to recover a (let's say euclidean) ...
0
votes
1answer
120 views

Is it possible to use seq2Seq models to predict HTML code from XML file?

I have XML file that describes some embedded components. So the file has different markups that correspond to different fields. The intention behind this project is to generate automatically UI ...
1
vote
0answers
207 views

Machine learning on JSON/XML/DOM data

What's the best approach for machine learning on deeply hierarchical JSON/XML/DOM documents (not counting text nodes)? Say I want to recognize and generate documents similar to a training set of ...
2
votes
0answers
141 views

Is there an extension of PCA for data embedded in hyperbolic spaces?

I'm working on a project where we are embedding data into an n-dimensional Poincare ball similar to this paper. However, we'd like to take the additional step of reducing this data to a 2-dimensional ...
0
votes
2answers
2k views

Learning image embeddings using VGG and Word2Vec

Background: In word2vec we pass in a one-hot encoding of our target word into a simple neural network which is trained to predict context words from a window around our target. We eventually take the ...
0
votes
1answer
33 views

How to calculate accuracy of identification using embedings and corresponded labels? [closed]

I'm trying to implement deep speaker embeding system and after getting voice embeddings I need to somehow calculate accuracy. But there is no mention in deep speaker paper about how they calculated ...
5
votes
1answer
264 views

What is the difference in the latent space of a variational autoencoder and a regular autoencoder?

Should VAEs be even used for non-generative tasks? If I were to use both models for embedding images, how would the embedding space differ on a structural level?
2
votes
2answers
2k views

How to use “IDs” as an input variable to a ML model?

I am trying to include a variable like "account number" which is an "ID" as a predictive variable for a logistic regression model. In fact there are several columns in my dataset that are "IDs" but ...
1
vote
0answers
321 views

feature embedding for categorical features

I'm training a model and among the features, I have the language of the users. Right now I have done one-hot encoding on the language feature. But I think it would make more sense to have the language ...
1
vote
0answers
87 views

Derivation of local linear embedding

Might be a trivial question, but how do I solve for the following constrained optimization problem that appeared in local linear embedding? $$\min_{w_1,\cdots,w_k} \|x-\sum_{i=1}^k w_i x_i\|^2 \text{ ...