Return to Question

This question relates to my previous query regarding differences in IRT parameter estimates (2PLM model) between the ltm package in R and the Mplus latent variable program which is linked here: 2PLM IRT modeling of rare event behavioral data: Why changing discrimination and difficulty values?

Now having obtained similar parameter estimates across platforms and having a sense of reliability to my estimates, I want to have a better grasp of what one considers 'plausible' discrimination and difficulty (threshold) estimate values in 2PLM IRT modeling. As is the case with any data, this will obviously hinge on sample size, response patterns etc. However, because I am dealing with unique maternal behaviors that are very rare-- even in a child protective setting-- many of my binary indicators have extreme cuts, in some cases with only a 1% proportion of occurrence (i.e., score = 1) on a sample size of N = 343. Despite this, all of my models successfully converge and the best loglikelihood is replicated--and in cases where the best LL is not replicated, contending solutions present essentially identical parameter estimates.

However, for some dimensions assessed, I get extremely large discrimination and difficulty estimates (e.g., well beyond the -3 to +3 prototypical scale of theta, or the latent trait under consideration) along with extremely large standard errors (in parens). Take, for example, data for the following binary indicators of two different latent traits:

Indicator 1: discrimination = 25.71(547.95), difficulty = .60 (1.44)

Indicator 2: discrimination = -.11 (0.45), difficulty = -30.58 (130.42)

Obviously, these are very large values for the discrimination parameter (indicator 1) and the difficulty parameter (indicator 2) and the SEs are even more extreme. Nevertheless, the models converge and given the inherent nature of the data, I am inclined to treat these values as accurate estimates of the sample under consideration. As to their degree of precision, this will have to await replication efforts but given the laborious nature of coding such data, I imagine it is likely going to be some time before another dataset of this nature and size is assembled. With all that in mind, I am interested in recommendations or 'best practice' guidelines in such situations assuming limitations of the data and precision of the estimates are openly acknowledged.

This paper on sparse data bias with maximum likelihood estimators was useful in many ways, but did not quite suit my needs in the current situation http://www.bmj.com/content/352/bmj.i1981

This question relates to my previous query regarding differences in IRT parameter estimates (2PLM model) between the ltm package in R and the Mplus latent variable program which is linked here: 2PLM IRT modeling of rare event behavioral data: Why changing discrimination and difficulty values?

Now having obtained similar parameter estimates across platforms and having a sense of reliability to my estimates, I want to have a better grasp of what one considers 'plausible' discrimination and difficulty (threshold) estimate values in 2PLM IRT modeling. As is the case with any data, this will obviously hinge on sample size, response patterns etc. However, because I am dealing with unique maternal behaviors that are very rare-- even in a child protective setting-- many of my binary indicators have extreme cuts, in some cases with only a 1% proportion of occurrence (i.e., score = 1) on a sample size of N = 343. Despite this, all of my models successfully converge and the best loglikelihood is replicated--and in cases where the best LL is not replicated, contending solutions present essentially identical parameter estimates.

However, for some dimensions assessed, I get extremely large discrimination and difficulty estimates (e.g., well beyond the -3 to +3 prototypical scale of theta, or the latent trait under consideration) along with extremely large standard errors (in parens). Take, for example, data for the following binary indicators of two different latent traits:

Indicator 1: discrimination = 25.71(547.95), difficulty = .60 (1.44)

Indicator 2: discrimination = -.11 (0.45), difficulty = -30.58 (130.42)

Obviously, these are very large values for the discrimination parameter (indicator 1) and the difficulty parameter (indicator 2) and the SEs are even more extreme. Nevertheless, the models converge and given the inherent nature of the data, I am inclined to treat these values as accurate estimates of the sample under consideration. As to their degree of precision, this will have to await replication efforts but given the laborious nature of coding such data, I imagine it is likely going to be some time before another dataset of this nature and size is assembled. With all that in mind, I am interested in recommendations or 'best practice' guidelines in such situations assuming limitations of the data and precision of the estimates are openly acknowledged.

This paper on sparse data bias with maximum likelihood estimators was useful in many ways, but did not quite suit my needs in the current situation http://www.bmj.com/content/352/bmj.i1981

This question relates to my previous query regarding differences in IRT parameter estimates (2PLM model) between the ltm package in R and the Mplus latent variable program which is linked here: 2PLM IRT modeling of rare event behavioral data: Why changing discrimination and difficulty values?

Now having obtained similar parameter estimates across platforms and having a sense of reliability to my estimates, I want to have a better grasp of what one considers 'plausible' discrimination and difficulty (threshold) estimate values in 2PLM IRT modeling. As is the case with any data, this will obviously hinge on sample size, response patterns etc. However, because I am dealing with unique maternal behaviors that are very rare-- even in a child protective setting-- many of my binary indicators have extreme cuts, in some cases with only a 1% proportion of occurrence (i.e., score = 1) on a sample size of N = 343. Despite this, all of my models successfully converge and the best loglikelihood is replicated.

However, for some dimensions assessed, I get extremely large discrimination and difficulty estimates (e.g., well beyond the -3 to +3 prototypical scale of theta, or the latent trait under consideration) along with extremely large standard errors (in parens). Take, for example, data for the following binary indicators of two different latent traits:

Indicator 1: discrimination = 25.71(547.95), difficulty = .60 (1.44)

Indicator 2: discrimination = -.11 (0.45), difficulty = -30.58 (130.42)

Obviously, these are very large values for the discrimination parameter (indicator 1) and the difficulty parameter (indicator 2) and the SEs are even more extreme. Nevertheless, the models converge and given the inherent nature of the data, I am inclined to treat these values as accurate estimates of the sample under consideration. As to their degree of precision, this will have to await replication efforts but given the laborious nature of coding such data, I imagine it is likely going to be some time before another dataset of this nature and size is assembled. With all that in mind, I am interested in recommendations or 'best practice' guidelines in such situations assuming limitations of the data and precision of the estimates are openly acknowledged.

This paper on sparse data bias with maximum likelihood estimators was useful in many ways, but did not quite suit my needs in the current situation http://www.bmj.com/content/352/bmj.i1981

This question relates to my previous query regarding differences in IRT parameter estimates (2PLM model) between the ltm package in R and the Mplus latent variable program which is linked here: 2PLM IRT modeling of rare event behavioral data: Why changing discrimination and difficulty values?

Now having obtained similar parameter estimates across platforms and having a sense of reliability to my estimates, I want to have a better grasp of what one considers 'plausible' discrimination and difficulty (threshold) estimate values in 2PLM IRT modeling. As is the case with any data, this will obviously hinge on sample size, response patterns etc. However, because I am dealing with unique maternal behaviors that are very rare-- even in a child protective setting-- many of my binary indicators have extreme cuts, in some cases with only a 1% proportion of occurrence (i.e., score = 1) on a sample size of N = 343. Despite this, all of my models successfully converge and the best loglikelihood is replicated--and in cases where the best LL is not replicated, contending solutions present essentially identical parameter estimates.

However, for some dimensions assessed, I get extremely large discrimination and difficulty estimates (e.g., well beyond the -3 to +3 prototypical scale of theta, or the latent trait under consideration) along with extremely large standard errors (in parens). Take, for example, data for the following binary indicators of two different latent traits:

Indicator 1: discrimination = 25.71(547.95), difficulty = .60 (1.44)

Indicator 2: discrimination = -.11 (0.45), difficulty = -30.58 (130.42)

Obviously, these are very large values for the discrimination parameter (indicator 1) and the difficulty parameter (indicator 2) and the SEs are even more extreme. Nevertheless, the models converge and given the inherent nature of the data, I am inclined to treat these values as accurate estimates of the sample under consideration. As to their degree of precision, this will have to await replication efforts but given the laborious nature of coding such data, I imagine it is likely going to be some time before another dataset of this nature and size is assembled. With all that in mind, I am interested in recommendations or 'best practice' guidelines in such situations assuming limitations of the data and precision of the estimates are openly acknowledged.

This paper on sparse data bias with maximum likelihood estimators was useful in many ways, but did not quite suit my needs in the current situation http://www.bmj.com/content/352/bmj.i1981

This question relates to my previous query regarding differences in IRT parameter estimates (2PLM model) between the ltm package in R and the Mplus latent variable program which is linked here: 2PLM IRT modeling of rare event behavioral data: Why changing discrimination and difficulty values?

Now having obtained similar parameter estimates across platforms and having a sense of reliability to my estimates, I want to have a better grasp of what one considers 'plausible' discrimination and difficulty (threshold) estimate values in 2PLM IRT modeling. As is the case with any data, this will obviously hinge on sample size, response patterns etc. However, because I am dealing with unique maternal behaviors that are very rare-- even in a child protective setting-- many of my binary indicators have extreme cuts, in some cases with only a 1% proportion of occurrence (i.e., score = 1) on a sample size of N = 343. Despite this, all of my models successfully converge and the best loglikelihood is replicated.

However, for some dimensions assessed, I get extremely large discrimination and difficult estimates (e.g., well beyond the -3 to +3 prototypical scale of theta, or the latent trait under consideration) along with extremely large standard errors (in parens). Take, for example, data for the following binary indicators of two different latent traits:

Indicator 1: discrimination = 25.71(547.95), difficulty = .60 (1.44)

Indicator 2: discrimination = -.11 (0.45), difficulty = -30.58 (130.42)

Obviously, these are very large values for the discrimination parameter (indicator 1) and the difficulty parameter (indicator 2) and the SEs are even more extreme. Nevertheless, the models converge and given the inherent nature of the data, I am inclined to treat these values as accurate estimates of the sample under consideration. As to their degree of precision, this will have to await replication efforts but given the laborious nature of coding such data, I imagine it is likely going to be some time before another dataset of this nature and size is assembled. With all that in mind, I am interested in recommendations or 'best practice' guidelines in such situations assuming limitations of the data and precision of the estimates are openly acknowledged.

This paper on sparse data bias with maximum likelihood estimators was useful in many ways, but did not quite suit my needs in the current situation http://www.bmj.com/content/352/bmj.i1981

This question relates to my previous query regarding differences in IRT parameter estimates (2PLM model) between the ltm package in R and the Mplus latent variable program which is linked here: 2PLM IRT modeling of rare event behavioral data: Why changing discrimination and difficulty values?

Now having obtained similar parameter estimates across platforms and having a sense of reliability to my estimates, I want to have a better grasp of what one considers 'plausible' discrimination and difficulty (threshold) estimate values in 2PLM IRT modeling. As is the case with any data, this will obviously hinge on sample size, response patterns etc. However, because I am dealing with unique maternal behaviors that are very rare-- even in a child protective setting-- many of my binary indicators have extreme cuts, in some cases with only a 1% proportion of occurrence (i.e., score = 1) on a sample size of N = 343. Despite this, all of my models successfully converge and the best loglikelihood is replicated.

However, for some dimensions assessed, I get extremely large discrimination and difficulty estimates (e.g., well beyond the -3 to +3 prototypical scale of theta, or the latent trait under consideration) along with extremely large standard errors (in parens). Take, for example, data for the following binary indicators of two different latent traits:

Indicator 1: discrimination = 25.71(547.95), difficulty = .60 (1.44)

Indicator 2: discrimination = -.11 (0.45), difficulty = -30.58 (130.42)

Obviously, these are very large values for the discrimination parameter (indicator 1) and the difficulty parameter (indicator 2) and the SEs are even more extreme. Nevertheless, the models converge and given the inherent nature of the data, I am inclined to treat these values as accurate estimates of the sample under consideration. As to their degree of precision, this will have to await replication efforts but given the laborious nature of coding such data, I imagine it is likely going to be some time before another dataset of this nature and size is assembled. With all that in mind, I am interested in recommendations or 'best practice' guidelines in such situations assuming limitations of the data and precision of the estimates are openly acknowledged.

This paper on sparse data bias with maximum likelihood estimators was useful in many ways, but did not quite suit my needs in the current situation http://www.bmj.com/content/352/bmj.i1981