What is a tensor? And why isn't a matrix a tensor?

Note that this is only a first draft, a first sketch, but you should get an idea ;-). 

A tensor is a physical quantity. In one dimension, temperature is a tensor. One dimensional tensors are called scalars (tensor of zero oder). But the temperature is independent of its representation. You can specify it in Kelvin or in degree Celsius. 

The same is in two or more dimensions (tensor of first order). Electrical fields are an example of a two or more dimensional tensor. But they do not depend on the coordiante system. An electcrical field may be represented in a Kardesian coordinate system or in a cylindrical coordinate system. So the componenets are different, but the pyhsical meaning is not. 

The electrical permitivity of a material is a tensor of second order. And it is again independet of the coordinate system. In a Kardesian coordinate system you may write it as a matrix. But it is only a representation, you cannot say that a matrix is a tensor.

A tensor maps a number of vectors (order of the tensor) to a value multilinearly.