I apply k fold cross validation using svms ( classification or regression, it doesn't matter) and I optimize my parameters based on the mean testing error. If I want to predict the value of one of my samples what do I do? I choose randomly one of the (perhaps best) k fold realizations where this sample happened to be on the testing set and predict its value?Or is it better if I apply the leave one out methodology in order to have one "testing phase" prediction for each sample?
Neither: you typically train a new model with the optimal hyperparameters on the entire training set and use that for prediction. Usually, it will perform marginally better than your cross-validation estimate because you are using more data for the final model.
An alternative that is sometimes used is to create an ensemble of the models you trained on each fold and then use the ensemble for predictions.