# Questions tagged [softmax]

Normalizing exponential function which transforms a numeric vector such that all its entries become between 0 and 1 and together sum to 1. It is often used as the final layer of a neural network performing a classification task.

181 questions
Filter by
Sorted by
Tagged with
1answer
21 views

1answer
45 views

### Large difference in accuracy for sigmoid vs softmax

I am experimenting on a neural network model I found on Kaggle for Titanic dataset where the problem statement is to determine whether a person has survived or not. The input I am providing is of this ...
1answer
24 views

### Number of features in multiclass Logistic Regression with categorical predictor

Assume that I want to predict a response with 3 classes. I have two features $X_1$ and $X_2$ where $X_1$ is continuous and $X_2$ is categorical with 5 categories. What would be the number of ...
0answers
20 views

1answer
124 views

### Is multinomial logistic regression really the same as softmax regression

Multinomial logistic regression (MLR) is an extension of logistic regression for more than $2$ classes. The extension is made up by keeping linear boundaries between classes and using the class $K$ as ...
0answers
42 views

### binary cross entropy vs multi cross entropy

i am new to neural networks I know that multi class entropy is same as binary class entropy when the categories are only (0,1), but can some one explain it mathematically with an example that ...
0answers
234 views

### Derivative of softmax function as a matrix

I have a generalised n-layer neural network. Currently, I am using it to perform digit classification (on the MNIST dataset), using a softmax + cross-entropy loss setup with simple stochastic gradient ...
0answers
84 views

### Softmax backpropagation

I know there are similar questions out there, but none of the answers really helped me. I'm working on an own neural network implementation and I want to implement the softmax activation function. I'm ...
1answer
36 views

### Number of parameters in sigmoid vs. softmax cross entropy

Assume I have a data point $\mathbf{x} = [x_1, x_2, \ldots, x_D]^\top$ which I want to classify into one of two mutually exclusive categories $\mathcal{C}_0$ and $\mathcal{C}_1$. I can create a simple ...
2answers
458 views

### Predicting proportions with Machine Learning

I am working on a machine learning problem where I have to predict a set of $N$ numbers (proportions) for each data point, all of them summing to one. One toy example to illustrate my problem would be ...
1answer
596 views

### Softmax derivative implementation [closed]

I know there are already multiple similar questions out there, but still don't really understand the derivative of the softmax function. That's how I implemented the softmax function in java: ...
1answer
133 views

### Is there a name for the composition of the cross-entropy and softmax functions?

This is a simple question but I'll give some background. The softmax function $S: \mathbb R^K \to \mathbb R^K$ is defined by  S(u) = \begin{bmatrix} \frac{e^{u_1}}{\sum_j e^{u_j}} \\ \frac{e^{u_2}}{\...
1answer
78 views

### Does there exist ReLu regression?

If softmax regression is multinomial logistic regression, is there anything called ReLu regression?
2answers
31 views

### Softmax: can't wrap my head around these values

I've got three simple classes each with some count values and I want to calculate the probability distribution. Column $B$ is the count and column $C$ is $exp(count)$. The last column then devides ...
2answers
506 views

### With Sigmoid activation and Softmax normalization with cross entropy, are we fitting distributions?

Let's consider I have a multi layer neural network that is doing multi class classification. So each input sample belongs to one on N classes. Now, lets say the last layer has Sigmoid activation ...