Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [algorithms]

An unambiguous list of computational steps involved in finding a solution to a class of problems.

1
vote
0answers
239 views

Efficient KNN implementation which allows inserts

Suppose I have multi-dimensional datasets, which have many vectors as data. I am writing an algorithm which needs to do k nearest neighbour searches for all those vectors - classical KNN. However, ...
2
votes
0answers
239 views

How does R calculate the acf() function as quickly as it does?

I noticed that R computes the acf(data,n) in order $O(n)$ time, not $O(n^2)$, so it cannot compute it brute force using double for loops. How does R calculate it ...
0
votes
1answer
50 views

Why W is transposed in the final form of multiplicative updates of NMF - $H^*=H * \frac{W^T\frac{V}{WH}}{W^T 1}$

For NMF using Kullback-Liebler divergence \begin{equation*} d_{\mathrm{KL}} (\mathbf{V}\ \vert\vert \mathbf{WH})\stackrel{\hbox{cst.}}{\hbox{=}} \sum_{ij}-V_{ij}\log\sum_{k}{(WH)_{ij}} + \sum_{ij}\...
4
votes
2answers
2k views

Batch reinforcement learning: Algorithm example

I have been reading the Reinforcement learning: An Introduction by Sutton and Barto (2012) and I have come across the batch learning method. Unfortunately, this method is not very well described in ...
2
votes
2answers
129 views

Grouping large groups into smaller groups while keeping people together

Let's say we have a grouping of people as such: ...
1
vote
0answers
40 views

Which Machine Learning Algorithm to Look into?

I have tried googling and couldn't find much of relevance so I have come here to ask. In essence, I want to know which machine algorithm can be used to (well, to some extent) detect repeating ...
1
vote
0answers
32 views

Number of possible permutations for a Weighted Kendall Tau Measure

The Kendall-Tau distance can measure distance between rankings. If we would like to measure the number of possible combinations that arises for a particular Kendall-Tau distance, it can be calculated ...
2
votes
1answer
806 views

Training data has categorical target variable, but I want to get numeric target variable for new samples

I hope I am asking the question to the right section of StackExchange. I have data which consists of 5 independent variables, and 1 dependent (target) variable. The independent variables are numeric, ...
0
votes
1answer
48 views

How to regularize parameters across a 2D array

I'm attempting to fit a parameter (which will be a 2D array) to an array of data which corresponds to spatial locations (i.e. longitude/latitude). The parameter can vary from point to point but I want ...
4
votes
1answer
1k views

How to use predicted features in prediction?

I have ratings data. My ratings data contains some technical features like the Originator (Channel), the exact day & hour of the broadcast, duration of the program etc. and, obviously, the label, ...
0
votes
1answer
49 views

Physical meaning behind the choice of the proposal distribution in Markov Chain Monte Carlo (MCMC) methods

Let us consider the conventions on names used in the theoretical derivation of Metropolis-Hastings Monte Carlo as outlined here, for the sake of common nomenclature. What we are building is a step-...
0
votes
2answers
38 views

Algorithm to cluster chart type

So, I have more than 20,000 entities. Each entities has their own data point (time series). Let say entities A1 to A20000. A1 has data point from year 1 to year 60. A2 has data point from year 5 to 60,...
0
votes
2answers
103 views

PCA when # of features larger than # of samples? [duplicate]

I'm using this code that adopts PCA for domain adaptation. I have some questions regarding how it handles the situation when the # of features is larger than the # of samples. Here is the snippet of ...
2
votes
0answers
26 views

Adaptable statistical power

In software development, unit testing is (basically) executing an algorithm with a fixed input and test if the output is as expected. Example : "test that MyFactorial(4) returns 24" where "MyFactorial"...
3
votes
1answer
156 views

How do you do belief propagation on nodes with conditional dependence?

Take this graph: $$ \overset{S}{\fbox{$+$}} \underset{\textstyle \nwarrow \!\underset{\overset{\scriptstyle X_2}{\textstyle\bigcirc}} {}} {\overset{\textstyle \swarrow \!\overset{\...
2
votes
0answers
17 views

Space-filling design algorithms from a discrete domain

Given $S = (X_1, \ldots, X_n)$, where each $X_i \in \mathbb{R}^2$, are there algorithms that produce $(X_{(1)}, \ldots, X_{(m)})$, where $m << n$ and the $X_{(i)}$ are sampled from $S$ to fill ...
1
vote
1answer
351 views

clustering for categorical data with one column for observations

I'm trying to cluster a dataset using 4 variables, all of which are categorical variables. I'd also like to include another numerical variable that's actually the number of observations of another ...
4
votes
1answer
10k views

How does the L-BFGS work?

The purpose of the paper was to optimize some parameters by maximizing the regularized log-likelihood. Then they calculate Partial derivatives. And then authors mention that they optimize the equation ...
2
votes
0answers
166 views

Elastic net for general GLMs (in particular the Gamma family)

I am trying to understand the state of the art of a Gamma GLM regression with an elastic net penalty because I need to recreate it in a SAS environment (pity me). To my knowledge, the only package, in ...
0
votes
0answers
1k views

Algorithmic complexity (big-O) of CART tree building

I have implemented the tree building part of CART, but I'm wondering if it can be optimized. According to this quora answer, the complexity should be $\mathcal O(v \cdot n \log n)$ where $v$ is the ...
0
votes
1answer
128 views

Approximating SVM using Perceptron

Suppose that we have a set of linearly separable data and this pseudocode of Perceptron: ...
2
votes
1answer
29 views

Quantifying relationship between items, given many groups of items

I'm having trouble researching this, or even writing this question, because I'm not sure what this is called and I'm not familiar with a lot of the terminology. The best I can do is an example. Let's ...
1
vote
1answer
171 views

How to derive BPTT algorithm

To gain an understanding of BPTT, I'm attempting to derive the formulas for BPTT by hand, but keep getting stuck. Here's what I have so far. Forward Pass 1 $a_0 = x_0 * u_0$ $b_0 = s_{-1} * w_0$ $...
0
votes
0answers
73 views

What is the fastest way to calculate the leading singular value of a very large matrix (10mln x 100k)?

I only know of the following power iteration. But it needs to create a huge matrix A'*A when both of rows and columns are pretty large. And A is a dense matrix as well. Is there any alternative to ...
1
vote
1answer
34 views

Optimization of random variate algorithm for t-distribution

Consider the following polar algorithm described by Bailey: Generate uniformly distributed variables $U, V$ in $(-1,1)$. Let $W = U^2 + V^2$. If $W > 1$ go to step 1. Otherwise deliver $W$. Now $...
3
votes
1answer
103 views

A variant of polar ratio of uniforms method for Cauchy variables

The Handbook of Monte Carlo Methods (page 107., Algorithm 4.27) presents a variant of the polar ratio of uniforms method for generating standard Cauchy distributed variables: The better known version ...
1
vote
3answers
89 views

Machine Learning that improves our ML algorithms?

I know very little about ML, all I know is what I read on Flipboard or watch on youtube. So from what I know I think ML is a series of algorithms based on statistics and evolution, such that they try ...
7
votes
2answers
779 views

How can I quickly detect cheating variables in large data?

Suppose We have a data set with millions rows and thousands columns and the task is binary classification. When we run a logistic regression model, the performance a lot better than expected, e.g, ...
4
votes
1answer
167 views

EM algorithm is always used for mixture copula

I understand the EM algorithm and know there are other optimisation algorithms. However, I have seen that with a mixture copula, EM algorithm always seems to be used. Is there any reason why EM ...
3
votes
1answer
498 views

Replay memory algorithm in Q-learning with neural network

I read a lot about replay memory, but did not find the material, where this method is described in detail. Tell me, please, do I understand the algorithm correctly? And if not, tell me, please, where ...
0
votes
0answers
58 views

Detection of sawtooth lines

I would like to ask for a piece of advice on the following problem. I have a set of points that can behave as a sawtooth-like function. I would like to develop an algorithm that could verify that the ...
0
votes
0answers
47 views

“Uniform” Clustering

I have the data $\{(y_i,\omega_i), i=1,\dots,n\}$, where $y_i \in \mathbb{R}_+$ the response and $\omega_i>0$ a weight. Fixed $K>0$ I want to determinate $y_1^*,\dots,y_{K-1}^*$ such that $$ \...
0
votes
1answer
44 views

Is there an easy algorithm for generating a random distribution within a range but skewed toward the ends?

Is there an easy algorithm for generating a random distribution within a range but skewed toward the ends? I guess I am looking for some random distribution $x$ that is parameterized with some ...
1
vote
1answer
833 views

Back propagation and and various cost functions

I want to understand and extract a codeable back propagation algorithm. I'm mostly pure ignorant coder and my math skills are very weak. And that what I want to go further with is an extendable ...
2
votes
1answer
149 views

Statistical analysis to test algorithm robustness

I'm testing the robustness of an image processing algorithm to measure certain feature of images. The output of the algorithm is simple: 1 if the feature is found, 0 if the feature is not found. An ...
3
votes
1answer
72 views

Machine learning algorithms “from the book” [closed]

The prolific combinatorialist Paul Erdős used to say that God keeps the most elegant proof of each theorem in "The Book". This inspired the following attempt to capture what some of the book might ...
-1
votes
1answer
479 views

Clara algorithm

Here is an algorithm for clustering that calls Clara. I did not understand the final step. I mean step 3, 4. For i ¼ 1 to 5, repeat the following steps: Draw a sample of 40+2k objects randomly from ...
0
votes
1answer
58 views

machine learning on data with lots of fluctuation

I have CSV files that contains data of Cache performance on a source with different workloads for a particular time period ! For each time interval data is recorded , It includes columns like ReadHits ...
0
votes
1answer
53 views

How can I rate the difficulty of a question in an adaptive learning platform?

I'm working on an adaptive learning platform with some friends for a project that helps students to prepare their university admission test. We provide different kind of practice sessions and we have ...
1
vote
0answers
65 views

An efficient algorithm for sliding mean deviation

I'm searching for a computationally efficient algorithm to calculate the sliding mean deviation from sample $x_a$ to sample $x_b$ belonging to a large set $x_0, x_1 ... x_n$ with the condition that $a$...
-1
votes
1answer
2k views

Classification and mixed categorical and numeric variables

I've been working a little with weka and so far I haven't made my own database to apply a classifier but I've tried to look at the already existing files and from what I've seen there is absolutely no ...
5
votes
2answers
2k views

what is the “learning” that takes place in Naive Bayes?

As I recall algorithms like nearest neighbor don't build a model based on training data and then apply that model to test data. It just takes each new instance and compares it to all the data to find ...
0
votes
1answer
111 views

what are mixed variables in data mining?

I read that neural networks, SVM and neuro-fuzzy don't support "mixed variables." So what are those exactly? Does it refer to mixed types (numeric and non-numeric)? And if so, does that mean the ...
0
votes
2answers
31 views

How to improve algorithm for determining binge usage on a service?

For a side project I am trying to determine if users in a dataset are either "binge" users or not. I have a dataset of user log data, where each row is a user event. Right now my algorithm is ...
8
votes
2answers
625 views

Policy and value iteration algorithm convergence conditions

Policy and value iteration algorithms can be used to solve Markov decision process problems. I have a hard time understanding to necessary conditions for convergence. If the optimal policy does not ...
0
votes
1answer
19 views

How do data mining classifiers behave when we add training samples to previously established training models?

I have yet to choose a classifier to get my model but I wanted to see which possible classifiers fit what I'm looking for. In fact, I need an algorithm to establish offline training this way: The ...
-1
votes
1answer
80 views

How do big companies apply machine learning? [closed]

I was wondering how recommendation on youtube work for example? How are the algorithms applied, because every user gets different recommendations depending on his location, his past liked videos etc......
1
vote
1answer
176 views

Classification vs regression machine learning?

While studying data mining methods I have come to understand that there are two main categories: -Predictive methods: classification Regression -Descriptive methods: Clustering Association rules ...
0
votes
1answer
40 views

Which algorithm/api/approach to use to solve this task (and if even possible)?

I have some idea that I would like to test and I think the ML would be great solution. But I'm a little bit lost how it could be implemented. The input is just a text, could be anything (but ...
4
votes
0answers
91 views

What is the feasibility to run linear model for large amount of data?

I am looking for empirical approximations and guidelines on how many operations are required to run a linear model on a given amount of data, or if it's even feasible. Let's assume we use QR ...