6

TLDR; In computers numbers are stored in finite slots of memory. For instance, an integer number in mathematics is whole number such as ...,-2,-1,0,1,2,3,... that can go in both directions from negative infinity to positive infinity. In a computer this number can be represented by a type such as int8_t (in C++) which spans from -128 to 127. The situation is ...


5

Floating point arithmetic is an approximation to arithmetic with real numbers. It's an approximation in the sense that all digits of a number aren't stored, but instead are truncated to a certain level of precision. This creates errors, because values like $\sqrt{2}$, which have an unending sequence of digits, can't be stored (because you don't have enough ...


4

It simplifies the notation to work with $$g(\mathbf{y}) = f(\mathbf{y}+\mathbf{x}) - f(\mathbf{x})$$ because (as you can readily compute) $$g(\mathbf{0}) = 0;\ \nabla g(\mathbf{0}) = \nabla f(\mathbf{x});$$ and $f$ is convex if and only if $g$ is. In particular, note that for any $0\le h\le 1,$ the convexity of $g$ means $$g(h\mathbf{y}) = g((1-h)\...


3

Because the inverse of a small number is large. The inverse of a Grammian matrix $K = Q\Lambda Q^T$ where $Q$ is the eigenvectors matrix and $\Lambda$ the eigenvalue matrix, is effectively the $K^{-1} = Q\Lambda^{-1} Q^T$. As such when we inverse a very small eigenvalue from the diagonal matrix $\Lambda$, we get a very large number in the inverse of it as ...


2

What you are looking for seems to be halfway between automatic model selection, and automated machine learning (AutoML). Automatic model selection has been around for a while, but is limited in scope. It is useful when you already know before hand what family of models you should use for your problem (e.g. Polynomial Regression), but not exactly which ...


2

I will carry out the steps in a way I hope is clear enough to indicate what assumptions must be made about $y,$ $p,$ and $\delta$ to justify the steps. When a function $y$ is a local minimum of a functional $\mathcal L,$ adding a sufficiently small multiple $h$ of a "test function" $\delta$ cannot decrease the value of the functional. (Usually a test ...


2

If the classes are linearly separable, there exists a hyperplane (on the same feature space) to separate them. When there is not, the classes are either non-separable or separated by other types of hyper-surfaces, e.g. if instead of a line, a parabola in 2D feature space can separate the classes, it's said non-linearly separable. Features having non-linear ...


2

There's nothing "bad" about having 100% accuracy on training sample. In fact, it is common practice in deep learning to start with building a model that is able overfitt a small subset of training set before proceeding further. We are talking about overfitting when there's a discrepancy between training performance of the model, and the performance when the ...


2

Hello guys! Sorry for my english. I undestand that so your problem is a question about model choise. You have to want the best test error, no the simility between train and test error. The minumum in the image shows your goal, to be sure about your decision, you can graph the errors versus complexity (depth pode, trees or min samples leaf). Indeed, the best ...


1

Is this correct that the number of samples and dimensional is refer to by the diagram below? In the diagram below it seems you have 150 samples (from the leftmost ID columns) and each sample has dimensionality 4. So you will have no problems computing LDA here as 150 >> 4. Is that LDA have problem if sample size (n) << dimensions (d) Yes, ...


1

One way to look at the data: x <- c(2, 4, 5, 6, 7, 8, 9, 9, 4, 5) y <- c(0, 1, 0, 0, 0, 1, 1, 0, 1, 0) stripchart(x ~ y, meth="stack", ylim=c(.5,2.5), pch=19) abline(h=1.5, col="green2") From the stripcharts, we can see that the data values for the two groups are not much different. If integer values given in x are rounded continuous variables, ...


1

While somewhat unlikely this phenomenon can indeed happen. The results from local explainer by LIME can disagree (on occasion substantially) with the results of the global model. Probably it is worth considering different kernel widths as well as checking the goodness-of-fit of the LIME explainer too. More details: LIME is training a model in the "...


1

Well the answer to the question is probably a matter of preference and depends on whether you want to specialize in some specific field (e.g. reinforcement learning) or you're aiming at having a more in-depth (but not limited to one subfield) view on machine learning. If it's the latter I would recommend you to look at the following two titles (both ...


1

They're unrelated. So-called multimodal machine learning operates on multiple input modalities. That is, input signals of different types (e.g. auditory and visual input). Multimodal distributions have probability density/mass functions with multiple modes (i.e. peaks). The existence of multiple input modalities doesn't imply anything about multiple modes in ...


1

I had a very similar scenario to this where I needed to flag when a pump's pressure time series exceeded a certain threshold, but the data was unavoidably EXTREMELY noisy. The solution involved several steps, which succeeded in triggering at very sensible locations, and I think the method will work very well for you. Apply a median filter. That is, for each ...


1

You can view vectorization parameters in exactly the same way as 'normal' hyperparameters. These are parameters you have to fix beforehand and find good ones by evaluating them on validation sets. The length of your text embedding for example regularizes the expressability of your model. Larger embeddings will give a model more freedom to fit the data ...


1

We have $$P(t_i=1|x_i)=\frac{1}{1+e^{-y_i}}\rightarrow P(t_i=-1|x_i)=1-\frac{1}{1+e^{-y_i}}=\frac{1}{1+e^{y_i}}$$ The closed form for both $t_i\in\{1,-1\}$ is: $$P(t_i|x_i)=\frac{1}{1+e^{-t_iy_i}}=\sigma(-t_iy_i)$$ The negative log-likelihood of the data is: $$NLL=-\sum_{i=1}^n \log P(t_i|x_i)=-\sum_{i=1}^n\sigma(-t_iy_i)=\sum_{i=1}^n\log (1+\exp(-y_it_i))$$...


1

If we abstract away all of the details of the algorithms, tree ensembles have simple structure: A function to initialize a tree A function to choose a split A function to determine when to stop building a single tree A function to determine when to stop adding trees to the ensemble for t in range(max_trees): i = 0 ensemble[t] = ...


1

Independent variable being binary instead of numeric doesn't change anything. Logistic regression's assumption is still valid, because in the end it is the assumption of the model, irrelavant to the fact that it is correct or not.


1

Cross validation is commonly used for hyper-parameter (HP) tuning or having a more stable test performance estimate. If you're to tune some HPs in your algorithm, case (b) definitely makes sense, though I'd advise an outer CV for the test since dataset is small. But, if there is no HP to optimize and you only want to evaluate the test performance, case (a) ...


1

Having a bagging fraction of $1$: While usually a sub-optimal case that might hint to over-fitting, having a large bagging fraction (i.e. use most, or all, of available data being used in each iteration) is not catastrophic. Especially if our sample is not very large. What is important is that we have a robust and clearly defined way of testing the booster's ...


1

"True distribution" is the distribution of your data, it doesn't have analytical form. Moreover, you write it as some kind of distribution that considers function of $X$, while this doesn't have to be the case. For example, you could use ice-cream sales to predict sunny weather, this doesn't mean that there's a causal relationship that makes the weather ...


1

Formally speaking, for the prediction of $y_{t_{16}}$ and onwards, there is no leakage because the standardisation does not include information that would be unavailable at the time of prediction. That said, the training does have some leakage in itself though as for example when we predict $t_{10}$ we use info from parts of the training set that occur ...


1

I believe the amount of features is too large. The model which you train might generalize badly on other "test" sets. It could be worth trying to reduce amount of features with PCA and try again.


1

The typical approach is to train on all the data (other than test) using the best found hyper-parameters. For example, in cross-validation, you don't have a single training/validation pair, so you'd naturally do the final training on all the training and validation data. On the other hand, sometimes, (like in neural networks) you need a separate validation ...


1

In general, splits are random, (e.g. train_test_split) which is equivalent to shuffling and selecting the first X % of the data. When the splitting is random, you don't have to shuffle it beforehand. If you don't split randomly, your train and test splits might end up being biased. For example, if you have 100 samples with two classes and your first 80 ...


1

It may depend on where the data came from and how it was exported. It's not uncommon that real world data is sorted in some manner. For example it could be sorted by: user id timestamp of the observation outcome of interest In each of these cases if you do a test/train split on the data without shuffling then you may have a different data distribution in ...


1

I think we can treat this as an instance of online (or incremental) machine learning. That is, given this new data use assume that the training data arrives in a continuous manner and we need to tune our hyper-parameters on the fly. In a way, we are doing a "warm start" in our problem. There are quite a few papers on the matter particular for SVM: Fast ...


1

Let $A$ be an $m\times n$ matrix with rows $\boldsymbol{a}_1^T,\dots,\boldsymbol{a}_m^T$. That is $$A= \begin{bmatrix} \rule[.5ex]{1.5em}{0.4pt} & \boldsymbol{a}_1^T &\rule[.5ex]{1.5em}{0.4pt} \\ \rule[.5ex]{1.5em}{0.4pt} &\boldsymbol{a}_2^T& \rule[.5ex]{1.5em}{0.4pt} \\ &\vdots& \\ \rule[.5ex]{1.5em}{0.4pt} & \boldsymbol{a}_m^T &...


1

Its not advisable to use a sigmoid as activation function for your multiclass-classification scenario. If you would have only two classes sigmoid would be fine since the output space of sigmoid is $\text{sigmoid}(x) \in (0,1)$ and therefore a valid probability. Now you can use this probability as the probability for the positive class and $1 - \text{sigmoid}(...


Only top voted, non community-wiki answers of a minimum length are eligible