# Is this a valid approach to testing a hypothesis about the relationship between two variables?

I am trying to test a hypothesis I have about consolidation in the real-world market for a certain machine. (My apologies in advance for obfuscating a bit here, but some of the data is proprietary and may not be redistributed.)

1) The market has two components: new machines sold and existing machines in use. Companies act as agents who sell new machines and service the old machines. Machines can be used for 10-15 years before becoming unfit for use. The pool of machines in use has risen steadily over the years.

2) The data show that over the decades the number of agents has fallen steadily, although over the same period the number of machines in use has risen steadily. If you divide the number of machines by the number of agents to get a "agents per machines in use" value, this has risen steadily over time. In effect, the industry has become more efficient: it sells and services more machines with fewer (but presumably larger) agents.

3) My hypothesis is that the driver of this increased efficiency has been the rise in the ratio of machines in use to new machines sold. My theory is that the increase in machines in use offered an opportunity to some agents in the form of increased service revenues. Those opportunities required a broader set of skills and more organisation than simply selling new machines, so some agents out-competed others. Over time - decades, in fact - the stronger agents drove the weaker ones out of the industry or consolidated them. This led to an increase in overall efficiency.

I want to find out whether there is indeed a relationship between market maturity and market efficiency. My approach to testing this hypothesis has been as follows. (a) Calculate the ratio of machines in use to new machine sales as a proxy for market maturity; the higher the number, the more mature the market. (b) Calculate the ratio of machines to the number of dealers operating in the market as a proxy for market efficiency.

This gave me 45 yearly data points for each of the two data series. I then made a scatter plot in Excel, with the machines per agent in use on the x axis and the ratio of machines in use to new machines sold on the y axis. (I planned to post the chart here but I don't yet have enough reputation to do include images in posts.) The linear trend line equation of the plot is y = 0.7277x + 6.528 with an R2 of 0.7488. This looks like a fairly strong relationship but not suspiciously high. However, I have no formal training in statistics, so I'm concerned that I may be making a gross methodological mistake.

Q. My question is... is this approach valid or irredeemably flawed? I have tried to normalise the data somewhat but I am still comparing a ratio and an absolute figure. Is there something else I should do?

(Another approach I considered was simply comparing the machines in use per dealer to the number of machines in the market, but the scales are different by a couple of orders of magnitude. I got a 45-degree line and an R2 of 0.99.)

I use Stack Overflow (mostly for R) but this is my first time on CrossValidated. I realise this is a question that may have a discussion element to it rather than the preferred "answerable question". Happy to edit the question as required.

-
I'm not sure that you have a pure efficiency measure, which very simplistically is producing the same outcomes for lower cost, or producing more outcomes for the same cost. You don't appear to be controlling for agent size, whether users are experiencing the same quality of service over time, being charged the same rates, etc. –  Michelle Jan 26 '12 at 17:48
I think those are excellent points. However, as is so often the case in the real world that data is not available to me I must work with what I have. Ultimately I am interested in gross efficiency at the market level in terms of number of agents and the interplay with industry consolidation. –  SlowLearner Jan 27 '12 at 0:04
@Henry made some excellent points in his answer. We're both warning you that you probably don't have the data to be able to test your hypothesis. In addition, because you don't have control over any variables, you're going to be looking at correlations and not be able to deduct causality. –  Michelle Jan 27 '12 at 0:25

A couple of questions:

1. You are comparing two ratios with common numerators, this can lead to spurious correlations.
2. Why are you using a ratio of machines in use to new machines sold, why not just use sales? Market maturity seems to be defined on the basis of sales, e.g. see here.
3. You don't seem to have measured your definition of efficiency: in your third bullet you include servicing in your definition of efficiency, but your measure ignores this component.
4. You mention that you have 45 yearly data points, but you ignore time as the ordering factor in your scatter-plot. If one of your key arguments is that market efficiency has occurred, that suggests that time needs to be incorporated into the analysis, because there is a timing component to efficiency - it occurs later in the product lifecycle rather than earlier, so you need to show the effect of time.
5. I'm not sure why you have an intercept term included in your model, as you would expect 0 machines to be have been sold if there were no agents (not 6.5), so the slope should pass through the origin. What happens to the results when you remove the intercept?
-
Michelle, thank you, these questions have given me useful points to ponder and study. I will take a step back, reconsider and take another shot at the analysis. –  SlowLearner Jan 27 '12 at 9:44
I'm pleased this was helpful. –  Michelle Jan 27 '12 at 18:01