What is physical meaning of inverse covariance matrix (or concentration matrix, precision matrix), and how to calculate it?

A covariance matrix is both real and symmetric, which means it qualifies for a spectral decomposition.  In other words, you can factor it into a product of an orthonormal matrix times a diagonal matrix (which variances along its diagonal) times the transpose of that same orthonormal matrix: C=U∗Σ∗UTC=U∗Σ∗UTC = U*\Sigma*U^{T} The inverse of this is fairly easy to find since each of the elements is easily invertible.  A property of orthonormal matrices is UT=U−1UT=U−1U^{T}=U^{-1} Therefore, C−1=U∗Σ−1∗UTC−1=U∗Σ−1∗UTC^{-1}=U*\Sigma^{-1}*U^{T}, where Σ−1Σ−1\Sigma^{-1} is identical to ΣΣ\Sigma except each of the non-zero values is replaced with its reciprocal.  Therefore, if you can diagonalize (i.e. factor) your covariance matrix, the inverse can be found inexpensively.  It also posses a pleasing analytical form. If you want to calculate the inverse numerically, then I would recommend a linear algebra package like LAPACK.  It contains both diagonalization codes as well as efficient routines to take inverses of symmetric matrices. I'm not sure I can provide a great physical interpretation of this matrix, but it does appear in various contexts.  For example, C−1C−1C^{-1} is a term in the exponential of a multivariate Gaussian distribution.

A covariance matrix can actually be decomposed into a sequence of linear transformations (rotation and scaling matrices): Σ=RSSR−1Σ=RSSR−1\Sigma = R \, S \, S \, R^{-1} where R is a rotation matrix and S is a scaling matrix. Inverting this covariance matrix thus corresponds to: Σ−1=RS−1S−1R−1Σ−1=RS−1S−1R−1\Sigma^{-1} = R \, S^{-1} \, S^{-1} \, R^{-1} If we define the transformation matrix T=RST=RST=R \, S, then Σ=TT′Σ=TT′\Sigma = T \, T' and Σ−1=T′−1T−1Σ−1=T′−1T−1\Sigma^{-1} = T'^{-1} \, T^{-1} In other words, if you interpret the covariance matrix from a geometric perspective, then the inverse of the covariance matrix has a clear and intuitive explanation. The inverse covariance matrix then defines how much the original data should be transformed in order to become white data. This is explained in more detail in http://www.visiondummy.com/2014/04/geometric-interpretation-covariance-matrix/

It would be better to call it the "surprise" matrix. That is, the ij term tells you how surprising it would be if the product of the residuals for i and j were positive (specifically, +1). If the off-diagonals of the covariance matrix tend to be positive (positive correlation), the off-diagonals of this matrix will be negative. That signifies that after you've already been surprised by a positive residual for value i, and added to that the surprise you have for a positive residual for value j, you have to subtract some surprise for the fact that after all you did expect them to go in the same direction.The conditional independence property makes sense in this view. If the i and k values are independent conditionally on j, then there is no "leftover surprise" to be had between i and k, after you've dealt with the surprise you'd have from the two individually and from their interaction with j.If this is the precision matrix of a multivariate normal, this "surprise" is precisely the negative of the log density (same as the likelihood, but since we're taking it as a function of the observations, we call it density and ignore the terms which are constant in the parameters). The more improbable the observations were, the more surprised we are.