Let’s use a simple example. My training set contains 5000 schools. For each school, there is a matrix. Each row of the matrix is a student, each column of the matrix is the grade of a subject (math, chemistry, biology, and physics). Let’s assume each school has the same number of students (1000), so each matrix is 1000 by 4. I also know which of these 5000 schools are public, which of these 5000 schools are private. Are there any machine learning algorithm that can take matrix as input and build a predictive model (without feature engineering)?
If the datapoint is a matrix it is always possible to unroll the matrix to a vector. But in your case it dosn't make sense.
I think the best way for you is to use basic feature engineering to create a vector for each school. Features could be simple - average grade in math, chemistry, etc. You can use quantile, min, max, percentage of highest grade etc. The private/public would be another feature.
For this setting you don't event need to assume constant number of students in each school.
I will suggest a way different from the one which was already given. You can append school information each to student and then model it. This way, each student entry has information about his marks and one column which tells you which school he is from. Instead of having 5000 matrices each with 1000 entries, you just have 5000000 entries as input now.