Python: Factor Analysis VS Autoencoder(keras) - Should their values be close I'm trying to compare FactorAnalysis with AutoEncoders.
My understanding is that AutoEncoders should be able to capture more of the the "non-linearity" of the data. 
My guess is that the loadings(FA._components) should somewhat match weights of the hidden layer of the keras autoencoder ? (just a better value) ?
Am I completely wrong in thinking  that -  or is there something wrong with my coding setup 
I've setup the code to have a number of 2 FACTORS(loading) and thus set the hidden-layer's number of neurons also to 2.
from keras.layers import Input, Dense
from keras.models import Model
import numpy as np 


from sklearn.decomposition import FactorAnalysis
from collections import defaultdict


# this is the size of our encoded representations
encoding_dim = 3

x_train=np.array([[2,-1,3],[2.5,-1.5,3.3],[2.2,-1.2,3.6]])

inl= Input(shape=(3,))

encoded = Dense(2, activation='relu')(inl)
decoded = Dense(3)(encoded)

# this model maps an input to its reconstruction
autoencoder = Model(inl, decoded)
autoencoder.compile(optimizer='adadelta', loss='mse')

autoencoder.fit(x_train, x_train,
                epochs=3000,
                batch_size=3)


#print(autoencoder.predict(x_train))


print("\nWeights of Hidden Layer ~=  Should it be the same as  FA-Loadings'ish ?\n")
print(autoencoder.layers[2].get_weights()[0])


fm = FactorAnalysis(n_components=2,max_iter=150).fit(x_train)

print("\nFA-Loadings ~=   Should it be the same as  Autoencoders-weights'ish ?\n")
print(fm.components_)

 A: There is no reason for these to be exactly the same.
The only similarity I can see in here is being used in dimensionality reduction and hidden variable estimation.
A FA has Gaussian assumption and usually uses a Gaussian likelihood and Expectation-Maximization for optimization and learning the factor matrices.
An Auto-encoder is totally a different story. It's a neural network, often with non-linearities, trained with gradient descent, often with the objective of mean squared error to reconstruct the input.
There's no reason whatsoever these two have to converge to the same solution, unless for a very specific problem where the optimum solution for both methods are exactly the same, which is quite unlikely.
Since the objectives and the optimizers (and even structures, considering nonlinearity in NNs) are not the same, there is no guarantee that the parameters learned (weights) should be the same, or even have similar properties.
Also judging based on value similarities is not a good measure. If you want to compare two methods, perhaps try to use measures like eigen values of the covariance or such, or run some experiments and compare the empirical results.
You could perhaps compare FA with PCA, with a prior, which indeed have similarities, but still not the same. Comparing FA with an AE is a bit out of context.
