In finance, economics and many other fields, observations in a matrix form are often observed over time. For example, several key economic indicators are reported in different countries every quarter. Various financial characteristics of many companies are reported over time. Import-export figures among a group of countries can also be structured in a matrix form. Although it is natural to turn the matrix observations into a long vector then use standard vector time series models, it is often the case that the columns and rows of a matrix represent different sets of information that are closely interplayed. We propose a novel matrix autoregressive model that maintains and utilizes the matrix structure to achieve greater dimensional reduction as well as easier interpretable results. The model can be further simplified by a set of reduced rank assumptions. Estimation procedure and its theoretical properties are investigated and demonstrated with simulated and real examples.