Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

New answers tagged

0

That’s what logistic regression is trying to do. Based on your predictors, $x$, the output models the conditional probability, $p(y=1|x)$. A random forest also tries to model the conditional expectation, $E[Y|X=x]$, which is also equal to this conditional probability since $Y$ is binary.


0

He is referring to a problem where you have a one output, a sigmoid neuron. When you initialize the weights of the network you can set bias to approximately $-2.3$. Why? The last layers bias is going to be dependant only on the statistics of the data, since it doesnt have any connection to the input. Meaning that if you have an imbalanced dataset, where ...


0

No, it is not possible. In order to use backprop you have to have non-zero derivatives of the function. Derivative of the step function is zero everywhere.


1

You are training a machine learner to identify patterns in your (training) data. If you later incorporate data it has never encountered before it has no idea what to do with it (it literally is seeing this information for the first time). Think of it as reading Chinese, while you've only learned English. Sure there's a lot of info in the Chinese text but ...


0

It depends on what do you think is going to be happening to the model after deployed for inference. Is it going to be fed the data only from those 15 countries? Then the first approach is a way to go. Or do you expect the data coming to the model to come from different countries than you already have? Then the second option is the way to go.


1

Typically a dense layer follows the LSTM/RNN layer(s), because the output of the RNN cell is of dimension of your choice, i.e. latent space dimension. Since you've three outputs, the final Dense layer will have three neurons, aiming to regress genre points. The RNN layer's duty is to figure out a compressed, latent representation of your series going back a ...


0

You touch several points: accuracy: you have to decide, what measure of accuracy is best for your case. It may be MSE, it may be R squared, it may be informedness. You have to ask yourself, how you would measure the distance from your regressors answer to the perfect answer. importance: luckily, feature_importance is already implemented in sklearn's random ...


0

It's not related to the "structure", it's related to the level of certainty that the relative count for a given case in your data is a correct estimation of its probability. (By "relative count" I mean the rate: Number of occurrences divides by total number of examples in the dataset.) Consider a dataset with a few features and a label that is "positive" or ...


0

Suppose your previous layer outputs hwd, 1*1 would take it to hwk where ideally k is less than d. The picture you are showing is of the inception module where the basic assumption is that features exist at many different scales. A filter of 3*3 cannot caputure a feature in a 5*5 window. And a 5*5 filter has a hard time modelling a 3*3 filter. So we try to ...


0

When the "positive class" is rare, attempting to make all-or-nothing classifications will usually be misleading and not very informative. It is far better to predict tendencies in such cases, and measures such as sensitivity, specificity, proportion "correctly" "classified", precision, and recall are improper accuracy scores that when optimized will result ...


0

Go for more epochs + add a batch-normalization layer as you added Drop Out layers. Both batch-normalization and Drop Out will regularize you net in order to generalize learned hyperplanes


0

I found a solution it is called dynamic topic modeling. I've linked an article documenting its' use. It is still undergoing research, but it's basically an LDA that takes time into account and can print topics change over time. https://github.com/rare-technologies/gensim/blob/develop/docs/notebooks/ldaseqmodel.ipynb Also check out Bleis' google talk on the ...


1

To me, this sounds like a problem where quantile regression could come in handy. The reason for this is that you need to compare distributions of model performance metrics (e.g., AUC) across levels of a factor such as country. To achieve this comparison, you could choose to compare certain quantiles of these distributions across the levels of this factor, ...


0

Perform a nested test using (generalized) linear models or ANOVA equivalently. Derive season as a fixed effect. Then fit the linear regression model adjusting for categorical season as a dummy variable with 3 levels. Perform the 3-degree of freedom nested test against the intercept-only model. If the result is statistically significant, conclude that mean #...


0

In order to identify a series as having "seasonality" one needs to consider both the presence of sarima structure and/or seasonal dummies. To do so one needs to identify and adjust for anomalies . To do so one needs to identify and adjust for possible power transforms or to use weighted least squares.To do so one needs to identify possible level/step shifts ...


4

I think of an acquisition function as describing the utility of the point to be evaluated next in the Bayesian optimization framework. To give more details, let's think about the general concept of Bayesian Optimization and the setting in which it is usually applied. Consider a black-box function $f$ which is expensive to evaluate and we want to find the ...


0

You are correct that separating out training and testing sets poses big problems with reasonably sized data sets. Bootstrapping, sampling with replacement from the data set on hand, is a well established way to accomplish what you want. It's based on the idea that the process of resampling from the data set on hand represents the sampling process that got ...


1

What you can do is something called nested cross validation. Instead of creating one train-test split you create K train-test splits (as you would in k-fold cross validation). For each of your K training sets you perform cross-validation to select hyper-parameters (this is called the inner CV loop) and then test on the test dataset (outer loop). You will ...


0

They differ in their approaches to tree generation: RuleFit first generates a boosted decision tree ensemble. That is: It sequentially grows trees on a pseudo response variable, where the pseudo response for each tree is corrected for the predictions of earlier trees. The amount of correction for earlier trees is controlled by the learning rate (.01, by ...


1

I worked with data scientists who do not know linear algebra. The field, which pretentiosly has "science" in its name, is so vast that there's something to do for everyone willing. It is somewhat similar to programmers not knowing electronics, and most of them have no clue. You can survive and even prosper without linear algebra but you will not be able to ...


1

I think linear algebra is absolutely essential to the data scientist. A card carrying data scientist should have more than just a comprehensive knowledge of what tools are out there, but he or she should also have the ability to compare them, to develop new tools, and, when a job simply can't be solved by any tool, be able to explain why. Linear algebra (...


2

To add to what @Sycorax commented: If you're satisfied with merely being a consumer of scientific software, you can skate by without much knowledge at all. But if you're curious and interested in really understanding what's going on when your script runs, knowledge of statistics, linear algebra, calculus and numerical optimization are essential; ...


0

You can try ensembling the three models and check what is the outcome of that. Because ensemble will increase the accuracy and take out the best out of three of these models. Also, there might be a possibility these models are overfitting so try checking it with validation set and if there's no overfitting then pick the highest scoring model and ensemble ...


1

Several considerations here pertain to the background upon which the question is based rather than the text of the question itself. As the say in Maine, "You can't get there from here." In simplest terms, one must first have shape information in order to test for shape, and there isn't any shape information to test, as follows. 1) Concentrations are not ...


0

What will go wrong with this approach? Your approach is just general statement of the problem. There is nothing wrong with it. It's just severely underspecified, as many models fit this description - for example both RNNs and Transformer architectures do that. Suppose we can take care of the infinite dimensionality above, assuming all sentences in all ...


2

The most important point stems from the confusion that the tilde $\sim$ implies a sampling operation. But $\sim$ does not imply that something is sampled, which is an algorithmic/computational operation. It indicates that something is distributed according to some distribution. Now, when we train a VAE, we want to get gradients of the ELBO. The form of the ...


0

You have some pretty obvious colinearity. This has been discussed here a lot (see the colinearity tag) and if you then have additional questions, ask a new question.


0

One of the best ways to check if the model is doing good is to build a graph between the training error and validation error. Ideally it should look something like this. Graph from LSTM based stock price prediction for Mitsubishi Stock (Nikkei Index) Kernel location As you can see, that validation error is not really improving with progressing epochs, with ...


1

assume we have n neurons and each neuron has the probability to be disabled. situation 0: zero neuron remains, n neurons are disabled, C(n,0) situation 1: only one neuron remains, n-1 neurons are disabled, C(n,1) situation 2: only two neurons remain, n-2 neurons are disabled, C(n,2) . . . situation n: n neurons remain, 0 neurons are disabled, C(n,n) so C(n,...


1

It sounds like might want to look into Pointer nets - the output is a set of pointers to different locations in the input. They’ve been used for things like finding boundary points for convex sets and ordering sequences of numbers. Even with Pointer nets (and basically any type of architecture, as far as I know) require sequence inputs to be padded to one ...


1

Two seminal publications on the early stopping in ML are: Zhang and Yu (2005) "Boosting with early stopping: Convergence and consistency and Yuan, Rosasco, and Caponnetto (2007) "On early stopping in gradient descent learning". Both are rigorous papers; they explore the matter under of early stopping within the context of gradient boosting and ...


0

You already outlined the most straightforward approach: put all the lagged values in the same row (observation). For instance, if you data set has columns: (x, y), and you think that in addition to contemporaneous values the past two values of column x can be useful, you create a new data set with columns (x(t),z(t),x(t-1),x(t-2)). As you mentioned the ...


0

I've been thinking about this for a while and just found the answer to my own question: When we replace 𝑡𝑎𝑛ℎ(𝑥) with 𝑡𝑎𝑛ℎ(𝑛𝑥) as an activation function we have changed nothing about how the activation function works. All we have done is rescaled all the weights of the network - which we are free to do arbitrarily. The only stage where $n$ should ...


0

First of all, the Random Forest cannot be applied to the following data types: images audio text (after preprocessing data will be sparse and RF doesn't work well with sparse data) For tabular data type, it is always good to check Random Forest because: it requires less data preparation and preprocessing than Neural Networks or SVMs. For example, you don'...


0

There are two distinct questions in the Kaggle page: "How has crime changed over the years?" For this you can simply plot some time based densities and histograms. "Is it possible to predict where or when a crime will be committed? Which areas of the city have evolved over this time span?" No. You might be able to predict aggregates, like monthly ...


1

A given point $x$ belongs to the closest centroid $K(x)$. This by definition means that $x$ is further away from all other centroids not $K(x)$. By adding a new centroid $K’$, $x$ will belong to it only if $|x-K’|\leq |x-K(x)|$. Thus, by writing down the definition of $D$, your result follows. Note that equality is achieved by placing the new centroid on ...


0

There are 2 ways to approach this problem: Convert each categorical features to several binary indicators, a process known as "one-hot encoding" Apply a transformation known variously as "target encoding" or "impact coding" that replaces the categorical feature with a numerical one. You should be able to use any of those terms to get you started in your ...


1

Yes, if the data is roughly linearly separable an rbf kernel might perform worse due to over-fitting, especially if the data is unbalanced. The rbf kernel is also more complex and computationally expensive so depending on fitting algorithm, e.g. smo or sgd, and stopping criterion, as well as the amount of hyperparameter optimization, the rbf kernel might ...


0

The evidence lower bound is a bound on the log probability of the data. But there is no straightforward way to compute the ELBO, since it requires taking an expectation over the variational posterior. Therefore we need a procedure to estimate the ELBO (more specifically we need some way to estimate the gradient of the ELBO so we can optimize it). The ...


1

The cited paper concerns detecting and testing interactions in a set of features. The general notion of an interaction concerns a derived term representing the product of two or more features. For instance the authors state: While in principle [Factorization machines] can model high-order feature interactions, in practice usually only order-2 feature ...


1

Random Forest models 'overfit' by definition, however this seldom has an effect on their predictive power. When you are passing a training sample down it's corresponding random forest model, the sample will end up in the exact same terminal nodes it ended up during training. Therefore, the above left plot depicts the deviation of the sample value from the ...


0

Further, to expand on already good answers, for any classification task beyond a bivariate one, using the regression would require us to impose a distance and ordering between the classes. In other words, we might get different results just by shuffling the labels of the classes or changing the scale of assigned numeric values (say classes labeled as $1, 10, ...


0

I have just finished making a tictactoe bot. The way I have set it up is that it learns from the moves that I make. So, if I'm playing a PlayervsAI or PlayervsPlayer game, every move that the winner makes is recorded and saved in a file. So this data is essentially 'moves that lead up to a win'. The format of this data is the 'state of the board + ...


1

There is a lot of other questions/issues here: what model did you estimate? did your model accurately classify everyone? did you perform feature scaling/standardization before estimating the model? what kind of feature importance did you estimate (or just what package/commands did you use), as there is more than one way to get feature importance? Also, you ...


0

The answer is not clear because 2*2=2+2. I’ll explain. Suppose that you had third dimension winter/summer. In this case for soft max you’d need 8 classes, while with logistics it’s only 6 states in total, I.e. $2\times 3$vs. $2^3$. That’s what he means by non exclusive. The problem is inherently non exclusive, and if you try yo model it as if it was ...


0

I'll answer your question by highlighting a problem which could occur quite often if you use a softmax classifier. In this case, the output of softmax will be a vector of 4 probabilities in decreasing order. The interpretation of the softmax output is that for a well performing model, the topmost probabilities are the ones that are most likely to be true. ...


1

I will recommend the following paper titled "Explaining Adaboost". It cleared a lot of doubt I had about the algorithm, what it does and loss minimization.


3

We have no idea. And the AUROC won't help you. A statistical model alone does not generate value. (Therefore, you cannot quantify how much more value an improvement in your model yields.) What generates value is decisions. Better decisions generate more value. If your improved model leads to better decisions, then it generates more value. But you might ...


1

nu SvM basically uses a parameter nu instead of C (which is used as a hyperparameter in case of linear SVM) as a hyperparameter for penalising incorrect classifications. Range here basically indicates the upper and lower limits between which our hyperparameter can take it's value. E.g. k is between 1 to N in case of Knn and lambda is between 10^-4 to 10^+4 ...


0

All ROC are plotted the same way except the authors may choose different variable on x-axis. The idea of s ROC is to run the identification rate from zero to 100% on y-axis by changing the detection threshold. Suppose that your algorithm produces the probability p of a hit, such as logit regression. Once you got p, you need to decide whether this p is high ...


Top 50 recent answers are included