Tagged Questions

EDA stands for "Exploratory data analysis". Developed by Tukey to contrast w/ *confirmatory* DA (the formal testing of hypotheses), EDA is typically concerned w/ describing data numerically & graphically to make the data easier to understand & to yield new insights.

learn more… | top users | synonyms (1)

2
votes
2answers
96 views

Which is correct way for regression line?

I have a set of data (some Frequencies per month, Var1 is the month): ...
4
votes
2answers
205 views

How to do exploratory data analysis to choose appropriate machine learning algorithm

We are studying machine learning via Machine Learning: A Probabilistic Perspective (Kevin Murphy). While the text explains the theoretical foundation of each algorithm, it rarely says in which case ...
0
votes
1answer
10 views

How to determine moving window size?

I am using moving window technique for data analysis... For example I compute the mean, the standard deviation and etc. for a given window. And I wonder if there's any good criterion to determine ...
4
votes
2answers
132 views

What are some good examples of exploratory data analysis today?

Are there some papers published which illustrate EDA used to tackle substantial data problems? I am particularly looking for actual (current) data examples, where plots have been made and statistics ...
1
vote
0answers
40 views

Can factor analysis improve the fit of a predictive regression model?

My company is working with a client who have built a logistic regression model to predict whether kids with psychiatric disorders will successfully complete a State intervention program (Yes or No). ...
3
votes
1answer
49 views

Is there a way to explain and generalise the decision made by random forest?

I saw a similar question was asked a few years ago, maybe there are updates on that. I would like to have a way to explain the decisions generated by random forest, possibly in a single tree. I ...
1
vote
3answers
97 views

I do not have hypotheses for my research and I have got only categorical data. what kind of data analysis can i conduct? also

I do not have hypotheses for my research and I have got only categorical data. what kind of data analysis can i conduct for correlation between variables? also is chi-square used only for hypotheses ...
3
votes
1answer
200 views

Determining probability distribution for datasets with missing values

As a part of my exploratory data analysis (EDA) prior to further analysis, I'm trying to determine a probability distribution of my pilot dataset's variables. A particular feature of this dataset is a ...
0
votes
2answers
32 views

Cross Vaidation in Factor Analysis

I am trying to conduct an EFA with a sample size of 150 respondents. I would also like to use cross-validation but my professor says that the sample is not big enough for that. Is that true?
1
vote
2answers
81 views

Is PCA appropriate for comparing subsets of panel data?

I have a large panel (5000+ subjects, 4 variables over 182 periods), and I've identified particular Granger-causal relationship in a large subset of those subjects (30% or so). I would like to somehow ...
2
votes
1answer
364 views

R scatterplot matrix with nonparametric density

I normally use MATLAB, or JMP but right now am working with R. I have ~150 dimensional data with a few hundred thousand rows. Some of the columns are non-informative, they only have one value. This ...
0
votes
0answers
38 views

Exploring properties of groups in R

I have a data frame with the following structure: ...
0
votes
0answers
31 views

exploratory analysis of associations between many variables

Can anyone suggest good references or examples of testing for associations between many (50-100) different variables/features (which, in my case, are primarily categorical or binary) This seems to me ...
0
votes
1answer
45 views

factor analysis: can irrelevant factors be identified by tweaking FA options?

So I ran a factor analysis (principal components method) on a dataset, first using the correlation matrix and then using the covariance matrix, both with varimax rotations. The results of both factor ...
2
votes
0answers
39 views

What to do when exploratory factor analysis results are different for complete-cases and imputed data?

I have a hundred items that I'm performing EFA on, with around 370 complete cases. Using parallel analysis to determine the number of factors to extract, EFA gave 9 factors, all of which make ...
0
votes
2answers
92 views

factor analysis problem in structural equation model

I am using an Extended Technology Acceptance Model and have adopted an instrument from a research paper. After performing factor analysis in SPSS, most questions related to different constructs ...
0
votes
2answers
137 views

should i take this variable which consist only 1 question in factor analysis? (all other variables are based on multiple questions)

I'm conducting a study on internet shopping adoption. I'm using a structural model from a research paper. My supervisor has asked me to conduct a factor analysis using SPSS. I'm quite weak in stats ...
2
votes
0answers
75 views

The (statistically unsound) exploratory study (no particular hypothesis)

It seems like a proposed study gets penalised if it doesn't also predict the direction of an effect. For instance, if I want to see the effect of a certain medication on happiness ratings, but do not ...
2
votes
1answer
72 views

Exploratory Analysis - finding the most important factor

I have a dataset of 113 variables. In exploratory analysis the first thing I want to know is what are the most important factors on a single variable (revenue). I learned that naive Bayes would ...
15
votes
5answers
253 views

How to keep exploratory analyses of large datasets in check?

When I start an exploratory analysis on a large data set (many samples, many variables), I often find myself with hundreds of derived variables, and tonnes of different plots, and no real way to keep ...
2
votes
2answers
41 views

What are some differences between confirmatory analysis and exploratory analysis?

In confirmatory analysis do you basically just test hypotheses? Then in exploratory analysis you try to generate hypotheses? In general, I know that you can first do exploratory analysis to form ...
1
vote
1answer
69 views

Exploratory factor analysis

I am running exploratory factor analysis and have extracted 3 factors, with items clearly loading on each one. However, it appears to me that the items that have loaded onto the first factor could be ...
0
votes
0answers
74 views

Does confirmatory = inferential, and exploratory = descriptive analysis / statistics?

Reading this webpage, I wonder: Are confirmatory analysis / statistics and inferential analysis / statistics the same concept? Are exploratory analysis / statistics and descriptive analysis / ...
6
votes
2answers
318 views

Is confirmatory vs exploratory statistics “induction vs deduction”?

This webpage says: Inferential Statistics - Deductive Approach Descriptive Statistics - Inductive Approach But I doubt it. If I understand correctly, Inferential Statistics is "given ...
5
votes
1answer
93 views

Is there a branch of statistics that tries to explain “why” the dataset has certain statistical properties?

Suppose I have a big dataset and I compute some statistical summary of it - e.g., the correlation of one dimension with another. I think a reasonable question to ask would be "what data points ...
3
votes
2answers
238 views

Exploratory data analysis vs null hypothesis testing

Why would exploratory data analysis be important to undertake before null-hypothesis tests?
6
votes
1answer
306 views

Variables lack correlation, but have pattern

Below is the graph of two variables, X and Y, each representing count data. N=348. Note the scales of the axes: Y is very approximately lognormal, but X has no decent fit (including Poisson, ...
0
votes
0answers
62 views

Multilevel exploratory factor analysis

We are creating a scale to measure the 'strength of tie' between two people. We conducted a survey where each person was asked to name a number of other people they know. They were then asked to ...
25
votes
5answers
829 views

Data “exploration” vs data “snooping”/“torturing”?

Many times I have come across informal warnings against "data snooping" (here's one amusing example), and I think I have an intuitive idea of roughly what that means, and why it may be a problem. On ...
2
votes
1answer
204 views

Combining Exploratory Factor Analysis with Random Forest for Binary Logistic Regression Feature Selection

For those of you familiar with Exploratory Factor Analysis (EFA) and Random Forest (RF), I have recently had an idea of combining these two methods to reduce the number of potential predictor ...
4
votes
2answers
127 views

Is analysis of existing data always exploratory, or can it be used for hypothesis testing?

This is a question about the rhetoric of describing analysis done using a public data set or any other pre-existing data set. Here is the hypothetical situation: A researcher reports that they have a ...
1
vote
2answers
101 views

How do I perform a multi-state decomposition with interaction effects?

I am trying to perform a decomposition with interaction effects. This paper provides a solution for n-factors where each factor has a binary state (see section 2). I have a problem with 2 factors, ...
1
vote
2answers
103 views

Implications of trying multiple tests for an analysis

Let's say I have a fixed set of data containing one variable. I'm interested in whether the variable shows a difference between two conditions. Now I perform a test and determine a p-value somewhere ...
0
votes
2answers
117 views

Unexpected factor structure in different samples

I have administered a questionnaire to respondents asking about customers' adoption of technology. I want to do an exploratory principal components factor analysis in Stata to form constructs from a ...
0
votes
1answer
79 views

EFA on R: Classifying Observations

I am newbie for R and Stats. I am using EFA in R [psych library] to identify the latent factors underlying my data. I got a reasonable amount of factors matching my purpose. My question is once I ...
2
votes
2answers
544 views

Can I do parallel analysis with any type of exploratory factor analysis/principal component analysis?

I wish to perform parallel analysis to determine how many factors I should extract from my maximum likelihood exploratory factor analysis. I have been referred to a program that calculates the ...
1
vote
0answers
51 views

Designing genomic search strategies for non-coding RNAs using BLAST

I am interested in developing a computational search of some types of non-coding RNAs (ncRNAs) on a new genome. I have some questions related to the nature of my experiment, because it is an ...
2
votes
1answer
163 views

Does the normal probability plot systematically underestimate the mean?

A normal probability plot is defined as a plot of $n$ pairs: ($[100(i-0.5)/n]$ th $z$ percentile, $i$th observation). Theoretically the points should fall close to a straight line with slope ...
0
votes
0answers
28 views

Finding kind of dependency

I have an interesting (to me) real-life problem but only a very basic understanding of statistics. I would appreciate a pointer to literature as I understand actually solving the problem is my ...
1
vote
1answer
150 views

Exploratory data analysis for discrete data

I am using a probit and a logit model for obtaining the choice probabilities of some data. What kind of plots can be useful to conduct a exploratory data analysis for these data? Here is the ...
4
votes
1answer
99 views

Analysis of binary variables

I have a data set consisting of about a quarter-million objects, each of which may have any of 30 particular features. So I might have Object 1: feature 3, feature 7 Object 2: feature 3, feature 29, ...
3
votes
2answers
123 views

Whether to exclude a case from two analyses when the case is only an outlier in one?

I have a question regarding the elimination of participant data for two statistical analyses on the same sample. Here is a bit of background information: I collected data from one sample (n = 181 ...
1
vote
0answers
90 views

Sample size for EFA

Looked but could not find so I apologize if this is a repeat of a question already asked - Can I conduct an EFA with an N of 180 and 70 variables? I say NO but my advisor still wants me to do it. ...
1
vote
2answers
1k views

Do data transformations before factor analysis need to be consistent across different variables?

(This question continues the previous one) I am creating a questionnaire, and I have identified 3 questions which are skewed (2 positively skewed & 1 negatively skewed). I successfully ...
5
votes
1answer
2k views

Are data transformations on non-normal data necessary for an exploratory factor analysis when using the principal axis factoring extraction method?

I am developing a questionnaire to measure four factors which constitute spirituality, and I would like to ask the following question: Are data transformations on non-normal data necessary for an ...
1
vote
1answer
5k views

Exploratory factor analysis - promax & factor cross-loadings

I have a question regarding the best practice for dealing with cross-loadings on factors after conducting an exploratory factor analysis using a promax rotation. Just to give a bit of background ...
8
votes
2answers
297 views

What are some good exploratory analysis and diagnostic plots for count data?

Does anyone know of good reference material on exploratory analysis and diagnostic plots for count data?
3
votes
1answer
138 views

When do I need a model?

Considering the scenarios of exploring data, predicting (in the range of predictors), extrapolating and explaining- for which would one need a model? When can one do without one? [Edit] By "model" I ...
7
votes
2answers
5k views

Difference between exploratory and confirmatory factor analysis in determining construct independence

Researchers often use two measures that have very similar items and argue that they measure different things (e.g., "I always worry when I am around cars"; "I am fearful of cars"). Lets call the ...
4
votes
1answer
85 views

Can complex sampling be ignored for exploratory patient-centered analyses?

Hello fellow StackExchange users, Preamble: I have been tasked with performing a cluster analysis (or possibly a latent class analysis, as I am pondering) to find non-overlapping groups of like ...