Is iterating LASSO a reasonable idea? Can Lasso regression be performed multiple times to systematically clean/remove parameters from a model? Would there be downsides to doing so/would it be considered poor practice?    
 A: You can try that. Doing so naively (i.e. setting hyperparameters only for each sequential fit, or using cross-validation for each sequential fit) seems likely to lead to increase the risk of overfitting. The way one could do this more appropriately is to do a cross-validation loop around the whole process, i.e. the number of sequential fitting steps, as well as the penalty parameters in each fitting step then become hyperparameters of an overall method that you'd want to tune overall rather than for each step in isolation.
A simple related idea is the relaxed LASSO, where you first do LASSO, but then combine the LASSO fit with a unconstrained fit of the model with the LASSO-selected terms. That's kind of a two-step version of what you describe, if the non-regularized fit gets all the weight (of course that's also only equivalent, if one decided for no regularization in the second LASSO fit of your approach).
A: I remember doing something like this to observe how the estimated shrinkage factors  $$ \left | \beta_{j}^{lasso} \right |/\left | \beta_{j}^{ols} \right |$$ evolves over iterations. I quickly tried to do another experiment for your question. As far as I understand, you want to run lasso regressions several times and discard the predictors for each iteration whenever they are assigned zero coefficients. If that was the question, here is an example for Boston data set with 101 iterations (First set of coefficients are estimated using ols, other 100 are 'lasso iterations')
library(ggplot2); library(zoo); library(MASS); 
library(glmnet); data(Boston)

dim(Boston) # 13 predictors
[1] 506  14 

# linear model
lm.boston <- lm(formula = medv~., data = Boston)

# defining the parameter list
params <- list()

# initialize lasso (first iteration)
optim.lambda <- cv.glmnet(x = as.matrix(Boston[, -14]), y = as.vector(Boston[, 14]))$lambda.1se

lasso.boston<- glmnet(x = as.matrix(Boston[, -14]), y = as.vector(Boston[, 14]), 
                  lambda = optim.lambda)

params[[1]] <- coef(lasso.boston)[-1,]

# start iterating over 2:100
for(i in seq(99) + 1) {
    coefs <- coef(lasso.boston)[-1,]
    pred.list <- names(coefs)[coefs != 0] # remove coefficients that are shrunk to 0

    optim.lambda <- cv.glmnet(x = as.matrix(Boston[, pred.list]), y = as.vector(Boston[,"medv"]))$lambda.1se

    lasso.boston<- glmnet(x = as.matrix(Boston[, pred.list]), y = as.vector(Boston[,"medv"]),
                  lambda = optim.lambda)

    params[[i]] <- coef(lasso.boston)[-1,] # collect estimated coefficients 
}

params <- c(list(coef(lm.boston)[-1]), params) # binding ols and lasso     estimations

# rbind'ing all coefficients
foo <- function (...)
# thanks to   http://stackoverflow.com/questions/21605401/why-does-rbindlist-not-respect-column-names
{
  dargs <- list(...)
  if (!all(vapply(dargs, is.vector, TRUE))) 
    stop("all inputs must be vectors")
  if (!all(vapply(dargs, function(x) !is.null(names(x)), TRUE))) 
    stop("all input vectors must be named.")
  all.names <- unique(names(unlist(dargs)))
  out <- do.call(rbind, lapply(dargs, `[`, all.names))
  colnames(out) <- all.names
  out
}

coef.path <- do.call(foo, params); coef.path[which(is.na(coef.path), arr.ind = T)] <- 0
autoplot(zoo(coef.path), facet = NULL) + geom_point()

Here are the parameter estimations over 101 models. As mentioned, first set of coefficients are obtained via ols and rest is via iterated lasso.

Model size over iterations
unlist(lapply(params, length)) # to see how many non-zero parameters are left in the model 

[1] 13 13  8  7  7  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3
[33]  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3
[65]  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3  3
[97]  3  3  3  3  3

If the plot is not sufficient, you can check what values the parameters take in each step using the params list.
I am still new to this kind of illustrations, so excuse my terrible formatting :-) 
Hope that helps.
@whuber
I wanted to add an example so the post owner could try it with his/ her own data. What I see from my own example is that iterating the lasso is actually not a good idea.
Considering the Least angle regression algorithm for Lasso solution from Prof. Tibshirani's page:


*

*Start with all coefficients bj equal to zero. 

*Find the predictor xj most correlated with y 

*Increase the coefficient bj in the direction of the sign of its correlation with y. Take residuals r=y-yhat along
the way. Stop when some other predictor xk has as much correlation
with r as xj has. 

*Increase (bj, bk) in their joint least squares
direction, until some other predictor xm has as much correlation
with the residual r. 

*Continue until: all predictors are in the model


Lasso is already an iterative algorithm that stops when each coefficient are penalized enough (and we already decide when to stop using cross-validation when frequentist approach is preferred). What I have done above is just to re-estimate by discarding variables at each step, which actually seems more like a  stepwise regression algorithm. 
A: It might create an underfit model, because the first model already has the optimal value of lambda through cross-validation.  Each subsequent round has a certain probability of rejecting a worthwhile predictor due to random roll of the cross validation folds, and because you've imposed an artificial variable selection (once a variable is gone, it's gone...) there's no way for the variable to get selected again.
You might consider bootstrapping instead.  Constructing lasso models for 100 bootstrapped samples of your data set should produce interesting statistics for the distribution of each coefficient and the fraction of samples where the variable was selected, akin to a p-value.  I don't know if there is a formal statistical validity for bootstrapping lasso, but even in absence of this, it should have practical value.
