# Tag Info

5

(PS) First of all I think Glen_b is right in his above comments on the usefulness of such a test: real data are surely not exactly Pareto distributed, and for most practical applications the question would be "how good is the Pareto approximation?" – and the QQ plot is a good way to show the quality of such an approximation. Any way you can do your test ...

5

The term inside your definition of f :- sum(-dexp(x,rate=theta,log=T)) is NOT the likelihood, but something else. What is it that is being calculated? When you consider what it is that is being optimized there, you will also understand why you're minimizing that function in order to maximize the likelihood. To quote your own algebra, here's the ...

4

In a general framework, profile likelihood intervals are approximate confidence intervals. The proof of this result is essentially the same as proving that the likelihood ratio statistic is (asymptotically) approximately distributed as a $\chi^2_k$ distribution. The idea consists of inverting the hypothesis test obtained from a likelihood ratio statistic. ...

4

There are an infinite number of ways for a distribution to be slightly different from a Poisson distribution; you can't identify that data is from a Poisson distribution. What you're talking about there by checking those three criteria isn't checking that the data come from a Poisson distribution by statistical means (i.e. by looking at data), but by ...

3

The function sim() randomly selects numbers are randomly chosen from a bivariate normal distribution with the specified vector of means and variance-covariance matrix in order to construct confidence intervals around the parameter estimates. The vector of means and the covariance matrix are determined by the ML estimates of the model parameters. Therefore, ...

3

Wikipedia has a good article on the topic, complete with formulas. The values in your matrix are the term frequencies. You just need to find the idf: (log((total documents)/(number of docs with the term)) and multiple the 2 values. In R, you could do so as follows: set.seed(42) d <- data.frame(w=sample(LETTERS, 50, replace=TRUE)) d <- ...

3

That model is qualitatively wrong in so far as it predicts negative values for Expenses for 22 out of 127 cases. This is a by-product of fitting a hyperplane to data for a non-negative response, which pays essentially no attention to the boundedness of the response, nonlinearity, or the side-effects of skewed distributions, including at least one outlier. ...

3

We can classify trend part of time series in two parts: 1) Deterministic; 2) Stochastic. Most of the tests around stationarity is to test the unit root (deterministic trend). In your case it is clearly deterministic trend. The trend part one can write as a function of time. Regarding the kpss test, null hypothesis of kpss test is the series is level or ...

3

(1) There is an extensive literature on why one should prefer full models to restricted/parsimonious models. My understanding are few reasons to prefer the parsimonious model. However, larger models may not be feasible for many clinical applications. (2) As far as I know, Discrimination/Discrimination indexes aren’t (?should not be) used as a ...

3

There is a package called "darch" http://cran.um.ac.ir/web/packages/darch/index.html Quote from CRAN: darch: Package for deep architectures and Restricted-Bolzmann-Machines The darch package is build on the basis of the code from G. E. Hinton and R. R. Salakhutdinov (available under Matlab Code for deep belief nets : last visit: 01.08.2013). ...

2

I will not give a complete answer (I have a hard time trying to understand what you are doing exactly), but I will try to clarify how profile likelihood is built. I may complete my answer later. The full likelihood for a normal sample of size $n$ is $$L(\mu, \sigma^2) = \left( \sigma^2 \right)^{-n/2} \exp\left( - \sum_i (x_i-\mu)^2/2\sigma^2 \right).$$ If ...

2

car does support the F-statistic for analysis of deviance tables: Anova(m, test.statistic="F") Resulting in Analysis of Deviance Table (Type II Wald F tests with Kenward-Roger df) Response: some_response F Df Df.res Pr(>F) some_factor1 0.30 1 3.4 0.6196 some_factor2 26.94 1 ...

2

If you are fitting a multivariate regression tree (MRT), then you need method = "mrt" - i.e. if you are using a matrix for the response, you can't use method = "class". If you just have a vector response, then if this is a factor and you use method = "class", then mvpart is doing nothing different to rpart i.e. the usual thing for a classification tree. If ...

2

It's certainly possible to find a solution - in fact, an infinite number of them! With a linear model in 1 variable, you can find a particular $x$ that will produce a particular $\hat y$, $\hat{y}_0$: $\hat y_0 = \hat{\beta}_0 + \hat{\beta}_1 x_0$ can be recast as $x_0 = (\hat y_0 - \hat{\beta}_0) / \hat{\beta}_1$ (Even when you have a nonlinear model in ...

2

You have a linear model with 3 predictors: $y=a_0+a_1x_1+a_2x_2+a_3x_3$ #example with the iris dataset mod <- lm(Sepal.Length ~ Sepal.Width + Petal.Length + Petal.Width, data=iris) coef(mod) #(Intercept) Sepal.Width Petal.Length Petal.Width #1.8559975 0.6508372 0.7091320 -0.5564827 Now, you want to fix one predictor to a specific value ...

2

When most people think of a non-parametric equivalent of ANOVA, they think of the Kruskal-Wallis test. The Kruskal-Wallis test cannot be applied to a factorial structure, however. The first workaround to this is to run all of your conditions as a one-way analysis. This does not let you test your factors individually, but you may be able to get what you ...

1

A simple way is to use the xtabs function, as in library("MASS") data(Insurance) xtabs(Claims ~ Group + Age, data = Insurance) R> xtabs(Claims ~ Group + Age, data = Insurance) Age Group <25 25-29 30-35 >35 <1l 67 70 56 346 1-1.5l 105 169 197 979 1.5-2l 46 124 153 540 >2l 11 41 47 200 You can ...

1

If you use type 3 for ANOVAs it is critical in R that you set the contrast to effect coding (i.e., "contr.sum"). The default contrast in R is dummy coding (or in R parlance, treatment coding) in which 0 represents the first factor level. This doesn't make too much sense when having interactions as explaind on the page I linked to. To set effect coding, ...

1

Interpretation of complex interactions is always tricky. The best way, I think, is to graph them. One way to start is with 3 graphs, one for each IV. For each of these graphs, one IV will be on the x-axis, and the the DV on the y-axis. Then make lines for the predicted value of the DV for each of several combinations of the other two IVs (e.g. quartiles). ...

1

While they work differently than your algorithm, I believe you'll find mob() and FTtree interesting. For Zeileis' mob see http://cran.r-project.org/web/packages/party/vignettes/MOB.pdf For FTtree,Gama's functional trees an implementation is available in Weka and thus RWeka. See http://cran.r-project.org/web/packages/RWeka/index.html for details.

1

You can compute the cluster assignments for a new data set with the following function: clusters <- function(x, centers) { # compute squared euclidean distance from each sample to each cluster center tmp <- sapply(seq_len(nrow(x)), function(i) apply(centers, 1, function(v) sum((x[i, ]-v)^2))) ...

1

This is another example of a widespread phenomenon, whereby significance tests from larger sample sizes give lower P-values for otherwise equivalent results. Consider a simpler example from tossing a coin, with two outcomes, heads or tails. Which of these is a stronger refutation of pr(heads) = 0.5: 7/10 heads, 70/100, 700/1000? Intuitively it should ...

1

The Kwiatkowski–Phillips–Schmidt–Shin 1992 paper proved an influential contribution to time-series analysis, because it was the first well-recognized test that tested for non-stationarity by taking as null-hypothesis that the series is (linear determinisitic) trend-stationary -contrary to the Dickey-Fuller (or Phillips-Peron) family of tests that have as ...

1

This is the implemented function (extracted from the C-sources; filennet.c, lines 156-165): static double sigmoid(double sum) { if (sum < -15.0) return (0.0); else if (sum > 15.0) return (1.0); else return (1.0 / (1.0 + exp(-sum))); }

1

You will need to know the differences between dataframes, matrices and lists. Also the differences between the R vector data types (logical, character, numeric, factor, list) and language types (formula, expression, function, call) and the situations under which they will be coerced to the next level in their type-hierarchies. (R is loosely typed and has a ...

1

Python is today very well suited for Data Analysis, Machine Learning and related fields. Please refer to packages such as: http://scikit-learn.org/stable/ http://pandas.pydata.org/ Answering the question, you should not invest time learning R for the moment if you already know Python, that's my opinion.

1

The usual approach would be to train a classifier on the resulting partitions (if possible, first clean the data, in particular fix any errors in the clustering). There is not much to be gained from mixing clustering and classification/prediction. Use clustering to produce an initial working hypothesis, refine this hypothesis, then use prediction to ...

1

I will not give an overly mathematical answer, but I would like to address your central question about the relationship between CI's and profile likelihood intervals. As the other respondents have pointed out, CI's can be constructed from a profile likelihood by using the $\chi^2$ approximation to the $normalized$ likelihood ratio. The accuracty of this ...

1

Best way would be to take photos of a few birds (together with the colour card) several times under each of a few different lighting conditions. Then model the measured brightness.bird.plumage with brightness.colour.chart as a continuous fixed effect, possibly non-linear, & individual.bird as a random effect. Significant interaction or heteroskedasticity ...

1

Let me emphasize that I'm a newcomer to clustering, and am not sure of the right answer in this case. That said, my first thought would be to fit a logistic model with random effects for family. Here is a tutorial from UCLA statistics on estimating these models in R.

Only top voted, non community-wiki answers of a minimum length are eligible