This sample size is rather small for Linear Mixed Model. Hox (2002) quotes Kreft's (1996) rules of thumb for minimal sample size:

 - 30/30 - minimum 30 groups with 30 observations per group
 - 50/20 - minimum 50 groups with 20 observations per group
 - 100/10 - minimum 100 groups with 10 observations per group

Ten observations per group is minimal sample size that is often mentioned. The general problem is that 1 or even 5 is a very small sample for a *group*. Also your general sample size of 170 is rather small for estimating as complicated model as multilevel model. The smaller the sample, the more biased your results can get.

An alternative you can use is to use Bayesian estimation, because it works well even with small sample sizes. However, with this approach you could end up with estimates being drawn purely from *prior* distribution.

You can find more information on preferable sample sizes and power analysis for multilevel modelling in those two books:

 - Snijders, T.A.B. & Bosker, R.J. (2012). *Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling*. London: Sage Publishers. 
 - Hox, J. (2010). *Multilevel Analysis: Techniques and Applications*. New York: Routledge.