Questions tagged [embeddings]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

Cycle detection on unsupervised time series data

i have some video data of production lines of some manufactories. In every video, an operator does the same 3-4 steps periodically for the entire video. Each periods of same steps is called cycle and ...
  • 1
0 votes
0 answers
11 views

Projecting data into CCA components

When I perform CCA and get the projected features. The eigenvectors for the Y data. Suppose I want to project new data into it? Do I do $NewData*Y_{compenents}$ or $Y_{componenets}*NewData$?
0 votes
0 answers
13 views

Recommender System: How to predict user ratings using Linear Regression and User/Item Embeddings?

I hope this is the right forum for this, but here goes: I am currently doing a capstone project for a course. Part of that is building a recommender system using various algorithms such as NMF, kNN, ...
0 votes
0 answers
17 views

does the ROC curve of a committee based predictor have any meaning?

would appreciate it if you'd take a moment to read the pipeline I've described below - it relates to how a learner that is based on a committee should be optimized w.r.t the threshold of the ROC curve....
  • 115
1 vote
1 answer
18 views

Algorithm to find closest document containing a set of strings, or variations of them

I have one dataset (A) containing several fields (strings) per sample. One of these fields is a name, and the others are all alphanumeric identifiers. I have another dataset (B) which contains highly ...
  • 333
1 vote
0 answers
26 views

Check if the embeddings preserve the true space structure

I obtained the embeddings from the pre-classification layer of a neural network for 600K instances. Each embedding's dimension is 1024, thus we have a sample of the embedding space, $\mathcal{X} \...
  • 261
2 votes
1 answer
32 views

Text similarity for badly written text

Consider the following scenario: Suppose two lists of words $L_{1}$ and $L_{2}$ are given. $L_{1}$ contains just bad-written phrases (like 'age' instead of '4ge' or 'blwe' instead of 'blue' etc.). On ...
0 votes
0 answers
13 views

embeddings for representing sequences in which items also have numeric attributes/features

My data is a collection of sequences of items (or events or webpages), in which each item also has associated numeric features. An example could be the following scenario: An on-demand movie ...
0 votes
0 answers
17 views

Updating temporal embeddings depending on the input

I'm building a forecasting model and I'm using a temporal embedding along with a positional embedding following the same architecture as Informer. ( https://arxiv.org/abs/2012.07436 ) My problem is ...
3 votes
1 answer
92 views

How is an embedding space optimized with respect to the loss function?

I understand that the point of the embedding layer is to reduce the dimensionality of the input space while also projecting it onto a space that represents the similarity between the medium in ...
  • 649
1 vote
0 answers
44 views

Information preserved in the kernel mean embedding

I have recently been introduced to the kernel mean embedding of distributions, that is the map $$\mu: \mathcal{M}^{1}_{+}(X) \rightarrow \mathcal{H} \\ \mu(P) := \int \phi(x) dP(x)$$ where $K$ is a ...
0 votes
0 answers
20 views

Word embedding to support OOV words for identity embedding

I want to create a model that performs a user-id embedding (hash of a user) for a Graph Neural Network learning task, the problem I am facing is that I might have a very large corpus of users, which ...
0 votes
0 answers
80 views

How to measure whether a set of vectors are more similar to each other than a set of randomly selected vectors would be?

I have ~15 000 word embedding vectors of length 256, that can be categorized into several groups (sizes 2 to 1500) via heuristic considerations. My aim is to measure whether the embedding vectors ...
0 votes
0 answers
15 views

Embedding extraction -> Classifier VS Embedding learning+ Classification on-the-fly?

I have two questions: How should we compare in general which of the following perform better? I have a graph and would like to perform a graph classification task. Is it better to extract graph ...
0 votes
0 answers
217 views

Triplet loss for text embedding and text similarity? [duplicate]

I am working on a triplet loss based model for text embedding. Short description: I have a database about online shop, I need to find the suitble product when users enter a text on search bar. I ...
0 votes
0 answers
71 views

How to improve the PMI (Pointwise Mutual Information) Quality for document based PMI

Generating word embeddings from the PMI is well understood and known to be equivalent to SGNS (skipgram negative-sampling) under certain conditions. I was able to get good quality word embedding using ...
user avatar
0 votes
0 answers
89 views

How to optimize K-means to eliminate outliers and unrelated clusters?

I clustered document embeddings with K-Means. Embeddings have 2048 dimensions. Now, i am trying to optimize clustering. There are two problems. 1- Some clusters may have outlier samples. 2- Sometimes,...
1 vote
0 answers
40 views

Model that gracefully handles missing values at inference

Say I have a neural network that predicts whether a restaurant will be successful in a given neighborhood and uses the following features: type of food served type of music played type of decoration ...
  • 529
0 votes
0 answers
115 views

Best encoding for a multiple-value categorical feature

I have a neural network classifier which works reasonably well, trying to predict city of destination given (among other things) city of origin. The city of origin is encoded into an integer form (e.g....
  • 529
0 votes
0 answers
44 views

How to train a custom embedding?

I have data with a lot of categorical features. The cardinality of some of these features is quite big (>100), so I want to avoid using one-hot encoding. The idea is to use an embedding. The ...
  • 313
1 vote
0 answers
19 views

Create score from various features/variables

I have 4 measurements for different sensors and I would like to somehow create a score for each measurement ranging from 1-10. I have tried performing PCA with 1 component on these 4 columns and then ...
  • 159
0 votes
0 answers
35 views

Using compositional data analysis to represent chemical compounds

I've recently got some insights about compositional data analysis, wondering whether it could be suitable for the framework I'm currently in. Recently, I've been very interested in trying to find some ...
1 vote
0 answers
49 views

Definition of the word "embedding"

The mathematical definition of the word "embedding" requires the mapping to be injective, so in that context one speaks of, for example, embedding real numbers in complex numbers (ie, ...
  • 856
4 votes
3 answers
761 views

Embedding data into a larger dimension space

Embeddings or latent spaces are vector spaces that we embed our initial data into that for further processing. The benefit of doing so as far as I am aware, is to reduce the dimension. Often data has ...
7 votes
1 answer
490 views

If the curse of dimensionality exists, how does embedding search work?

The curse of dimensionality tells us if the dimension is high, the distance metric will stop working, i.e., everyone will be close to everyone. However, many machine learning retrieval systems rely on ...
  • 33.9k
0 votes
0 answers
25 views

Word-embedding does not provide expected relations between words

I am trying to train a word embedding to a list of repeated sentences where only the subject changes. I expected that the generated vectors corresponding the subjects provide a strong correlation ...
1 vote
1 answer
38 views

Generating embeddings for languages without a written representation?

I'm considering the topic of generating an NLP Embedding for a language without a written standard or a significant corpus. I realized that as challenging as that is, it is still not as challenging as ...
2 votes
0 answers
69 views

Is there an MDS/embedding algorithm that is more suitable to the goal of clustering a graph

I am testing ideas on clustering a particular graph. After testing a set of graph clustering/community detection algorithms I thought about mapping the graph to a vector space and using vector space ...
4 votes
1 answer
150 views

ArcFace - How to compute $\cos(t+m)$ when $t+m > \pi$

I am trying to understand the ArcFace Implementation and I am stuck at one condition. If the $ \cos(t) > \cos(\pi -m)$ then $t + m > \pi$. In this case the way how we're computing $\cos(t+m)$ is ...
  • 141
1 vote
0 answers
74 views

Learn sentence embeddings from a sequence of token embeddings

I want to build a sentence classifier that takes the sentence as a sequence of token embeddings. I'm specifically interested in the methodology for learning the sentence embedding from the sequence of ...
  • 111
1 vote
1 answer
150 views

Extracting word embedding features of a sentence using Transformer-XL

As you know, there are several pre-trained models that we can use to extract word embeddings. As an example, I can use the following codes to retrieve word2vec features of my text: ...
  • 385
1 vote
0 answers
310 views

How to pass an array of variable length to the input of the neural network?

I have a bunch of two-dimensional points that I want to feed into my neural network as an input. Those points are positions of the visible obstacles around my agent. The main challenge is that the ...
  • 111
1 vote
0 answers
87 views

Embeddings model: out of sample prediction with Keras for Collaborative Filtering

I have been trying to play with an example on Collaborative Filtering for Movie Recommendations (keras.io), which builds embedding layers for movies and users. Now, in a regular pre-trained word- and ...
  • 11
1 vote
1 answer
200 views

What's the best practice for dealing with OOV characters?

I have read on the advantages of using character-level language models over word-level ones. In particular, you don't have to deal with the problem of out of vocabulary (OOV) words, since characters ...
1 vote
1 answer
94 views

What is the relationship between embeddings and deep metric learning?

Embeddings and deep metric learning seem architecturally identical. They both rely on using some hidden layer's vector representation of an input. What is the difference between the two? Are ...
  • 961
5 votes
1 answer
1k views

What is embedding? (in the context of dimensionality reduction)

In the context of dimensionality reduction one often uses word embedding, which seems to me a rather technical mathematical term, which rather stands out compared to the rest of the discussion, which ...
  • 1,800
1 vote
0 answers
2k views

Which dimensionality reduction technique works well for BERT sentence embeddings?

I'm trying to cluster hundreds of text documents so that each each cluster represents a distinct topic. Instead of using topic modeling (which I know I could do too), I want to follow a two-step ...
2 votes
0 answers
239 views

dimension of input layer for embeddings in Keras

It is not clear to me whether there is any difference between specifying the input dimension Input(shape=(20,)) or not ...
  • 31
0 votes
0 answers
63 views

Is there an algorithm for placing 2-dimensional embeddings into a grid so they can be displayed?

I’m using PCA to reduce images down to 2d embeddings and I’d like to display the images in a grid. The Pudding did something like this with book covers, using tsne and a library called RasterFairy, by ...
  • 113
1 vote
0 answers
102 views

Why are we interested in gradient with respect to input?

I am learning about sampling methods for Deep Embedding Learning. I was reading an article named: "Sampling Matters in Deep Embedding Learning" (https://arxiv.org/abs/1706.07567). In the ...
  • 11
2 votes
1 answer
589 views

What is the difference between embedding in pure math and embedding in ML?

In ML the term "embedding" gets tossed around a lot and the term basically means the construction of a function that takes a high-dimensional vector to a low-dimensional vector in such a way ...
1 vote
2 answers
40 views

Why are character level models considered less effective than word level models?

I have read that character level models need more computation power than word embeddings, and this is one of the major reasons for their less effectiveness, but i got curious because the word ...
0 votes
1 answer
60 views

Why do Dense layers perform better than a mix of Conv Layers, Recurrent Layers on Sentiment Analysis with BERT emebddings?

I have used BERT to make embeddings out of the imdb review dataset and I am trying out some models to check their perfomance on sentiment analysis (0 for the bad reviews and 1 for the good ones). I ...
  • 101
2 votes
1 answer
165 views

Is there any paper about applications of Deep Metric Learning on regression problem?

I'm trying to solve a problem in the field of transfer learning, more specifically, domain adaption where both the source domain and target domain are labeled. Basically it's to predict the ...
  • 133
2 votes
1 answer
1k views

Neural network backpropagation to update inputs, not weights (e.g. fine-tuning embeddings)?

I recently re-read Stanford CS231N lecture notes on computer vision and backpropagation, and I came across this passage (emphasis mine): Note that (as is usually the case in Machine Learning) we ...
1 vote
0 answers
53 views

Adam converges while SGD does not improve at all

I am trying to build a model based movie recommendation system with a neural network. The architecture looks as follows: ...
  • 11
7 votes
2 answers
2k views

Why BERT use learned positional embedding?

Compared with sinusoidal positional encoding used in Transformer, BERT's learned-lookup-table solution has 2 drawbacks in my mind: Fixed length Cannot reflect relative distance Could anyone please ...
3 votes
1 answer
689 views

What are state of the art methods for creating embeddings for sets?

I want to create embeddings in $R^D$ for sets. So I want a function (probably a neural network) that takes in a set $ S = \{ s_1, \dots, s_n \} $ (and ideally of any size, so the number of elements ...
1 vote
3 answers
60 views

General mathematical definition of a score

I understand what scores are in PCA, in particular this answer gives a good mathematical formulation: (Scores) are projections of the centred data in the linear space defined by the eigenvectors. ...
0 votes
1 answer
200 views

Embedded markov chain example

I have an example of in my textbook of an "embeded markov chain", where I don't understand one step. Suppose that $(X_n)_{n\geq 0}$ is Markov$(\lambda, P)$. $\lambda$ is the initial distribution and ...
user avatar