What you've outlined is probably the single most common error that machine learning researchers make when analyzing financial data: it's trivial to discover that a great predictor of tomorrow's price is today's price.
The statistical term of art for this phenomenon is "non-stationarity." We have a number of questions about how to test for the stationarity of a time series. One such thread is How to know if a time series is stationary or non-stationary? In the particular case of time series analysis of financial data, it might be helpful to review a high-quality statistical text, such as Statistics and Data Analysis for Financial Engineering, Second Edition (David Ruppert & David S. Matteson). On page 308, we find the remark
As mentioned, many financial time series do not exhibit stationarity, but often the changes in them, perhaps after applying a log transformation, are approximately stationary.
(This is a quite extensive textbook about time series data and financial data, so it's worth reading in some detail if you're interested in how to pursue this project further.)
So to answer your question, the example neural networks that you mention discover that the financial data are non-stationary, and these models make use of that fact when making predictions. But if you look at returns, then the non-stationarity phenomenon disappears, and the model is not able to discover such a simple rule to exploit.
The cure, in some sense, is to discover what drives stock prices, either generally or in the specific case of the equities you're studying. The price changes every second -- why is that? What information could a person have that causes a 0.1% shift from minute to minute, or 1% day to day? It's unlikely that yesterday's price movement, or the price movement the day before, will tell you much of anything about tomorrow's price movement by itself with a high degree of precision -- because, as we know, past performance is no guarantee of future returns.
Framed in this way, the problem is not about choosing a certain kind of neural network, but instead making a neural network that has relevant data to inform its predictions. So, right now, you know that a good predictor of price tomorrow is the price today. To improve on that, you'll have to find timely information that improves upon the "best guess" provided by yesterday's price data.
As an example of what form this information might take, consider pairs trading. In the 1980s, Morgan Stanley quants invented "pairs trading" and the strategy was profitable for a while. The premise is that two highly correlated stocks will tend to move together, so if there is movement in one that's not present in the other, you can make a trade with thesis that eventually the two stocks will return to their equilibrium. So your neural network would use information about one stock to place trades on the second stock, and vice-versa. Naturally, pairs trading is only profitable as long as the premise that the pairs are strongly correlated is true.