# All Questions

136,547 questions
293 views

### Can we compute bivariate from marginal distributions?

Can we do that? If yes, then what are the conditions which should be met?
4k views

### What to make of explanatories in time series?

Having worked mostly with cross sectional data so far and very very recently browsing, scanning stumbling through a bunch of introductory time series literature I wonder what which role explanatory ...
191 views

### Incremental one-way ANOVA

Imagine you have a set of four elements (A-D) with some numeric values of a measured property (several observations for each element): ...
290 views

### Principal components of spatial variables

I have a number of rasters of environmental data (~10) which may be important predictors for modelling species presence and abundance at ~10 different locations. I would like to know which of the ...
1k views

### How to quickly sample X if exp(X) ~ Gamma?

I have a simple sampling problem, where my inner loop looks like: v = sample_gamma(k, a) where sample_gamma samples from the ...
540 views

### Equations in the news: Translating a multi-level model to a general audience

The New York Times has a long comment on the 'value-added' teacher evaluation system being used to give feedback to New York City educators. The lede is the equation used to calculate the scores - ...
5k views

### Algorithm to dynamically monitor quantiles

I want to estimate the quantile of some data. The data are so huge that they can not be accommodated in the memory. And data are not static, new data keep coming. Does anyone know any algorithm to ...
15k views

### Why are fitted.values not part the R object returned from arima?

Starting out with arima models in R, I do not understand why fitted.values (of an AR(2) process for example) are not part of the output like they are in regressions. Did I miss them when running ...
243 views

### The usage of Newton's method in nonlinear regression

Would you please give an intuitive illustration of Newton's Method, when we deal with nonlinear regression? Basically I understand that if we can use Taylor's theorem to expand the RSS function of ...
9k views

### How to create coloured tables with Sweave and xtable? [closed]

I am using Sweave and xtable to generate a report. I would like to add some coloring on a table. But I have not managed to find any way to generate colored tables with xtable. Is there any other ...
418 views

### Do two variables need to be independent in order to obtain a correlation? [closed]

I would like to find the correlation between two variables. It was suggested to me that two variables should be independent; otherwise it is not meaningful statistically to calculate a correlation. ...
96k views

### When is it ok to remove the intercept in a linear regression model?

I am running linear regression models and wondering what the conditions are for removing the intercept term. In comparing results from two different regressions where one has the intercept and the ...
3k views

### What correlation coefficient and graph is appropriate with this data?

I'm not in Statistics field. I conducted the case study and collected the data as shown below I have data as shown in the table below: I would like to find correlation coefficient from this two table ...
23k views

### What is tied data in the context of a rank correlation coefficient?

I am not in statistics field. I have seen the word "tied data" while reading about Rank Correlation Coefficients. What is tied data? What is an example of tied data?
1k views

### Gaussian kernel estimator as Nadaraya-Watson estimator?

I'm working on a problem from "The Elements of Statistical Learning" (prob. 6.8): Suppose that for continuous response $Y$ and predictor $X$, we model the joint density of $X, Y$ using a ...
37k views

### What are disadvantages of using the lasso for variable selection for regression?

From what I know, using lasso for variable selection handles the problem of correlated inputs. Also, since it is equivalent to Least Angle Regression, it is not slow computationally. However, many ...
19k views

### Understanding the k lag in R's augmented Dickey Fuller test

I played around with some unit root testing in R and I am not entirely sure what to make of the k lag parameter. I used the augmented Dickey Fuller test and the Philipps Perron test from the tseries ...
3k views

### Scrambling and correlation in low discrepancy sequences (Halton/Sobol)

I am currently working on a project where I generate random values using low discrepancy / quasi-random point sets, such as Halton and Sobol point sets. These are essentially $d$-dimensional vectors ...
2k views

### How can I compare Likert scale data of two clusters in SPSS?

In SPSS, I want to compare two clusters of management sciences department faculty members in two universities. Which test should I use? Can you explain how to do it in SPSS?
4k views

### How are zero values handled in lm()?

I am working on a linear regression with R and there are many 0 values in my predictor variables. How are these handled in R's lm() function? Should I remove this ...
2k views

### Efficient way to classify with SVM

I'm doing a binary classification using SVM classfier, libsvm, where roughly 95% belongs to one class. The parameters C and gamma are to be set before the actual training takes place. I followed the ...
6k views

### Mahalanobis distance between two bivariate distributions with different covariances

The question is pretty much contained in the title. What is the Mahalanobis distance for two distributions of different covariance matrices? What I have found till now assumes the same covariance for ...
1k views

### Detecting patterns of cheating on a multi-question exam

QUESTION: I have binary data on exam questions (correct/incorrect). Some individuals might have had prior access to a subset of questions and their correct answers. I don’t know who, how many, or ...
1k views

### How to calculate confidence interval when only a part of the samples are valid?

I will simplify our problem in this way. Say there are 100,000 cases in total to examine. Due to the time limitation, we randomly selected 2,000 of them. Then we found 1,000 of them are invalid, so we ...
2k views

### Generate random strings based on regular expressions in R

I'm trying to generate random textual data based on regular expressions. I'd like to be able to do this in R, as I know that R does have regex capabilities. Any leads? This question has come up ...
4k views

### Complex regression plot in R

I need to draw a complex graphics for visual data analysis. I have 2 variables and a big number of cases (>1000). For example (number is 100 if to make dispersion less "normal"): ...
949 views

### Best methods of feature selection for nonparametric regression

A newbie question here. I am currently performing a nonparametric regression using the np package in R. I have 7 features and using a brute force approach I identified the best 3. But, soon I will ...
738 views

### ROC surfaces in R

If my response variable has 2 levels, 0 and 1, I can use a ROC curve to assess the accuracy of my predictive model. But what if my response variable has 3 levels, -1, 0, and 1? Is there a way to ...
146 views

### Max-margin clustering with size constraint

Given a dataset $D$ and a distance measure, I want to split the dataset into two disjoint subsets $X, Y$ of a specified size (say 80% and 20% of the original size), so that the minimum distance of all ...
950 views

### How to transform data to have a new mean and maximum value?

I have data relative to image intensity, which range from 0 to 255. For some models I need to run, I must transform this intensity data to a different scale, which must have a certain mean (for ...
26k views

### Fast ways in R to get the first row of a data frame grouped by an identifier [closed]

Sometimes I need to get only the first row of a data set grouped by an identifier, as when retrieving age and gender when there are multiple observations per individual. What's a fast (or the fastest) ...
3k views

### How to compute prediction error from Relevance Vector Machine and Gaussian Process Regression?

Does anyone know how to construct a confidence interval for predicting a new test value given a trained Relevance Vector Machine (rvm) and/or Gaussian Process ...
6k views

### Interpreting range bars in R's plot.stl?

I have trouble figuring out what the range bars in plot.stl exactly mean. I found Gavin's post on this question and read the documentation as well, I understand ...
338 views

### Discussing binomial regression and modeling strategies

Today I have got a question about binomial/ logistic regression, its based on an analysis that a group in my department have done and were seeking comments upon. I made up the example below to protect ...
628 views

### What statistical technique would be appropriate for optimising the weights?

Background: I have the following data (an example): ...
241 views

### Building a classification rule for geographical mapping of cell phone number

I have a database of some 10000 cell-numbers with mapping of geographical areas they are associated to. I believe the first 5-6 characters in the numbers can point to the geo. area the cell number ...
718 views

### What is the power of the Ljung-Box Test for auto-correlation?

How large sample size should be for Ljung-Box statistic to achieve a power $\ge 0.5$ when number of lags tested is 1? How does the power fall ( assuming an AR(1) process), with increasing number of ...
10k views

### Visualizing a million, PCA edition

Is it possible to visualize the output of Principal Component Analysis in ways that give more insight than just summary tables? Is it possible to do it when the number of observations is large, say ~...
3k views

### Justification of one-tailed hypothesis testing

I understand two-tailed hypothesis testing. You have $H_0 : \theta = \theta_0$ (vs. $H_1 = \neg H_0 : \theta \ne \theta_0$). The $p$-value is the probability that $\theta$ generates data at least as ...
22k views

### Calculating proportions by age in R

I working on this data set with marital status and age. I want to plot the percentage of never married man versus each age. Could you please help me to figure out the way how to do it in R? So far I ...
5k views

### Best way to put two histograms on same scale?

Let's say I have two distributions I want to compare in detail, i.e. in a way that makes shape, scale and shift easily visible. One good way to do this is to plot a histogram for each distribution, ...
2k views

### Percent error or percent difference?

Using either percent error or percent difference, I want to compare one of my measured values from a set to the mean of the set. By reading the Wikipedia article on percent difference, it's still not ...
10k views

### In regression analysis what's the difference between data-generation process and model?

In regression analysis what's the difference between 'data-generation process' and 'model'?
176 views

### Feature selection for classification, controlling for sub-population

I have a bunch of points that belong to one of population P1, P2, ... Pn AND to class A or B. Within each population I'll be doing classification between A and B, and I want to select features that ...
790 views

### Derivation of distance in TwoStep clustering

I am working with the twostep cluster process in SPSS Modeler (Clementine) and trying to get a sense for the distance function used. It is a log-likelihood function (as stated in docs) but I am unsure ...
2k views

### How to generate user-friendly summaries of cluster analysis?

I used BIRCH and HAC to do clustering on my data. I want to now what type of information I can include in reports that my users can generate to get more insights on the clusters. I would have to dumb ...
2k views

### Computing confidence intervals for prevalence for several types of infection

I have a dataset about a population in hospital, and what type of infections patients have. Let say the number of patients is 100, 10 of them have pneumonia ...
6k views

### How to do ANOVA on data which is still not normal after transformations?

I'm looking at the effect defeat and entrapment inducing conditions have on subjective ratings of defeat and entrapment at three different time points (among other things). However the subjective ...
For a paper, I need an example of a dataset $(x_i,y_i)$ where the residuals are $iid$ (the $x$ do not represent time) with a discontinuity on the effect of $x$ on $y$. I have already found a dataset ...