Two level analysis, but mostly just one case per higher level - what to do? There are 500 stores (n = 500), for which I want to model revenue.
Each store has data on their level, like size and age (the lower level) but also data on a higher level (let's say city level, meaning: for the city where the individual store is located).
Does it make sense to use a multi level regression model, if there is mostly just one store per city in my dataset? Let's say I have 475 cities with 500 stores, meaning that only very few cities have more than one store and then it is just two stores for one of those cities.
Is it not better then, to just "pretend" every variable is on the store level and use a standard multiple linear regression with just one level?
 A: I don't think this question has a really good/canonical answer. Because of the low level of replication within cities, you don't have a lot of information to distinguish within- and between-city variation. You could:


*

*fit a multi-level model and hope for the best, i.e. that you don't run into numeric problems or singular fits

*pretend that the within-city variability is very large and treat every store as independent (this is what you suggest in your last sentence, i.e. running an ordinary regression model)

*pretend that the within-city variability is zero and average the observations within each city (you'd also have to average their covariates)

*pick some reasonable intermediate value for the relative degree of within- vs between-city variation and fit a multilevel regression with some parameters fixed

*pick a reasonable distribution of the within/between-city variance decomposition and use a Bayesian approach to average over the distribution


If this is a very important question, you should consider (a) running more than one of these analyses and see how big a difference it makes; (b) simulate data with characteristics that resemble your real data and see how the different methods perform. (The danger of (a) is that it encourages cherry-picking, i.e. you might be tempted to choose the method that gives you the result you like the best.)
