Questions tagged [scalability]
The scalability tag has no usage guidance.
12 questions
1
vote
0
answers
60
views
How to estimate the time and memory requirement for Sparse and variational Gaussian Process (SVGP) with minibatch approach
I'm implementing Sparse and variational Gaussian Process (SVGP) using GPflow library using minibatches. I've been trying to search for details regarding the time and memory complexities e.g. from this ...
- 5,907
5
votes
2
answers
248
views
What are some uses of logistic regression at scale?
Many libraries that scale linear and logistic regression assume a tall-skinny design matrix (many samples, few features), but I don't understand why you would need billions of samples if your data has ...
- 205
1
vote
0
answers
25
views
handling multiple time series through common model?
I have 1.5 lac/ 150 K timeseries . These are divided by geo locations. I have total 32 geo locations.Customer is expecting to have minimum number of model for all the 1.5 lac forecasting. How should i ...
- 1,195
2
votes
1
answer
113
views
Bayes and Naive Bayes code implementations
I know that Bayes classifier assigns the new data point $\pmb{x}$ to the class $\omega_j, \ j=1,\dots,M$, when
$p(\omega_j \mid \pmb{x}) = \max_{q=1,\dots,M}p(\omega_q \mid \pmb{x})$,
where
$p(\...
- 33
0
votes
1
answer
612
views
Persistent Cluster ID's for DBSCAN
When executing the DBSCAN algorithm over multiple runs on similar data (but not the same), I would like to generate persistent ID's so we can monitor how the clusters changed over time.
Selection of ...
- 1
1
vote
1
answer
86
views
Scalable machine learning for bigger data
I am aware of the theory of stochastic gradient descent, which is a faster way of developing linear regression. Through this we can have an 'optimized implementation' of linear regression. There are ...
- 1,124
21
votes
1
answer
494
views
How can we simulate from a geometric mixture?
If $f_1,\ldots,f_k$ are known densities from which I can simulate, i.e., for which an algorithm is available. and if the product $$\prod_{i=1}^k f_i(x)^{\alpha_i}\qquad \alpha_1,\ldots,\alpha_k>0$$ ...
- 108k
1
vote
2
answers
610
views
Best Scalable Classification Algorithms
I have a very large data set that I want to perform classification tasks on. There are about 40 million instances, 16 features, and 2 classes.
I'm attempting to use SciKit-learn ...
- 113
0
votes
1
answer
52
views
Scalability comparison with the help of regression
I created an algorithm and I tested it against a current algorithm.
The results are in this form:
...
- 121
4
votes
0
answers
518
views
How does t-SNE slow down with increasing number of dimensions?
I'm trying to understand the computational bounds of t-SNE. It's learned with SGD, so it'll have to go through some number of gradient-descent iterations. We can ignore that here, and focus on the ...
- 179
3
votes
2
answers
2k
views
Bisecting K-means using Dynamic Time Warping
I'm trying to cluster time series of different length and I came up to an idea to use DTW as a similarity measure, which seems to be adequate, but the thing is, I cannot use it with K-means, since it'...
- 2,877
6
votes
4
answers
2k
views
Solving a practical machine learning problem
I am currently doing my Phd in computational biology at Stanford. I get the data I need to answer the questions I am interested in. The data sets are sometimes "large" and these large problems take ...
- 2,637