Kalman filter parameter estimation From what I've known about Kalman filter, it requires all the parameters of the underlying state space model. Say the state space model is:
$$\xi_{t+1} = F\xi_t + v_{t+1}$$
$$y_t = H\xi_t + w_{t}$$
where $v$ and $w$ are disturbances.
Kalman filter needs the $F$, $H$, $Q$ (the covariance matrix of $v$) and $R$ (the covariance matrix of $w$) as well as $\xi_1$ as the initial state and the corresponding $P_1$ (the mean squared error of $\xi_1$) to start the recursion.
However, these parameters generally have to be estimated by numerical methods. Assume $y_t$ is Gaussian, how can the $H$ be estimated if we don't have any external knowledge about $\xi_t$? Alternatively, if we have some external knowledge of $H$, the estimation of all the parameters can be proceeded as well. What is the general rule to determine the initial parameter values or the state vector if we don't have any external knowledge?
 A: Everything you will ever need regarding estimation of parameters in a state space model is in this document:
https://cran.r-project.org/web/packages/MARSS/vignettes/EMDerivation.pdf
Kalman Filter/Smoother assumes that the parameters are known in advance so that the unobserved state can be estimated. The initial values for the state can be user supplied or a diffuse initialization approach can be used.
For this the standard reference is:
https://www.amazon.com/Time-Analysis-State-Space-Methods/dp/019964117X
For the matrix parameters and noise co-variance estimation the standard procedure is Expectation Maximization which is described in detail in the first reference, and also in
https://www.stat.pitt.edu/stoffer/tsa4/tsa4.pdf
chapter 6.
Archived version of tsa4.pdf (Time Series Analysis and Its Applications): https://web.archive.org/web/20210401070804/https://www.stat.pitt.edu/stoffer/tsa4/tsa4.pdf
A: In the usual state space model, the only things that are estimated are the state and its variance-covariance matrix (at each time point) - whether filtering or smoothing only changes what information you're conditioning on, but either way you end up with estimates of those things.
The $H$'s (and $F$, $Q$ and $R$ in your notation) are known/set/measured exactly/pre-specified, not estimated; it's part of your model for how observations are related to the state vector (and how state vectors evolve over time, etc).
$H$ is akin to the predictors in a regression model in that sense (i.e. that you don't estimate the $X$ matrix in regression)
A: As mentioned in other answers, you need values for the parameters in all system matrices ($F$, $H$, $Q$) in order to run the Kalman filter. However, you may have a state-space model with unknown parameters that you need to estimate. 
In order to do that, you may use the Kalman filter: running the Kalman filter with arbitrary values of the parameters will produce, as a byproduct, the likelihood. You can then embed the Kalman filter in an optimizing routine which tries different values so that the likelihood is maximized. Answering to other queries I have given detailed examples using package dlm in R. (Alternatives are packages MARSS and KFAS, among others.)
A: To address the question of initial values, I would suggest you read Time Series Analysis by Durbin and Koopman (2012) who go into great detail on the exact diffuse initialisation procedure (this is implemented in the R package KFAS). Which is probably what you want to use in the case of a non-stationary model. For a stationary model, you can easily enough solve for the steady state and initialise with that. For a mixed model, and it's a little more tricky and often overlooked - suggest you refer to Doan (2010) Practical Issues with State-Space Models with Mixed Stationary and Non-Stationary Dynamics
A: Other answerers mentioned EM, which is the most traditional approach. There are many others, however.

*

*you can do direct likelihood optimization, as various others have also remarked: run filter/smoother, obtain likelihood, iterate/optimize. Gradient expressions are available, see, e.g., Bayesian filtering and smoothing by Simo Särkkä, Chapter 12. (I have implemented this in my personal little Matlab Kalman filter/smoother toolbox, I'm sure it's also available elsewhere.) Notice that in this approach, you can use arbitrary combinations of unknown parameters, exploit known structure elements if you know part of or the functional form of one of the unknown matrices, impose constraints on the optimization scheme, etc.

*you can also exploit other gradient-based optimization schemes; see, e.g., Fitting a Kalman smoother to data (Barratt, Boyd; 2020) for an approach using a proximal gradient method; python toolbox available here.

The optimization problem can, in general, be non-convex. You can ignore that and hope for the best and/or use some kind of global optimization strategy, use multiple starting points, use some stochastic optimization scheme, etc. If you do have some reasonable prior guess about possible parameter values, exploiting that in the initialization and/or providing appropriate constraints can greatly help with convergence.
