{"id":71,"date":"2017-11-29T02:08:57","date_gmt":"2017-11-29T02:08:57","guid":{"rendered":"http:\/\/lop.blogwyrm.com\/?p=71"},"modified":"2017-11-29T02:08:57","modified_gmt":"2017-11-29T02:08:57","slug":"masked-scientific-arrays","status":"publish","type":"post","link":"https:\/\/lop.blogwyrm.com\/?p=71","title":{"rendered":"Masked Scientific Arrays"},"content":{"rendered":"
\n
\n<\/div>\n
\n
\n

Define a typical scientific array¶<\/a><\/h1>\n

The definition of the matrix A is designed to represent a typical scientific data set that varies by orders of magnitude. In addition, some of the data is invalid due to either no measurement ($$A_{0,1} = 0$$) or there was a measurement glitch ($$A_{1,2} = -1$$).<\/p>\n

For sake of completeness, the entire matrix is given by:<\/p>\n

$$A = \\left[ \\begin{array}{ccc} 1\\times 10^{-4} & 0 & 1\\times 10^{-5} \\\\ 1\\times 10^{-3} & 5\\times 10^{-3} & -1 \\end{array} \\right]$$\n<\/p><\/div>\n<\/div>\n<\/div>\n

\n
\n
In [16]:<\/div>\n
\n
\n
\n
<\/span>A<\/span> =<\/span> np<\/span>.<\/span>array<\/span>([[<\/span>1e-4<\/span>,<\/span>0<\/span>,<\/span>1e-5<\/span>],[<\/span>1e-3<\/span>,<\/span>5e-3<\/span>,<\/span>-<\/span>1<\/span>]])<\/span>\r\nprint<\/span> A<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
\n
<\/div>\n
\n
[[  1.00000000e-04   0.00000000e+00   1.00000000e-05]\r\n [  1.00000000e-03   5.00000000e-03  -1.00000000e+00]]\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n<\/div>\n
\n
\n
A simple matrix plot (using matplotlib’s pcolormesh) doesn’t show anything useful since all the ‘good’ numbers are orders of magnitude smaller than the bad.  Note also that the matrix is ‘plotted up’ as the zeroth row corresponds to the lowest row of the plot<\/p>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
In [17]:<\/div>\n
\n
\n
\n
<\/span>plt<\/span>.<\/span>pcolormesh<\/span>(<\/span>A<\/span>)<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
\n
Out[17]:<\/div>\n
\n
<matplotlib.collections.QuadMesh at 0x862fab0><\/pre>\n<\/div>\n<\/div>\n
\n
<\/div>\n
\n\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n<\/div>\n
\n
\n
Of course, the way to bring out the data is by taking a logarithm – but the inclusion of the ‘0’ and the ‘-1’ are going to cause problems.  Numpy gives two warnings (different ones) and it comes back with different values.  The resulting matrix should look like<\/p>\n
$$ log_{10} (A) = \\left[ \\begin{array}{ccc} -4 & -\\infty & -5 \\\\ -3 & -2.3 & nan \\end{array} \\right] $$\n<\/p><\/div>\n<\/div>\n<\/div>\n
\n
\n
In [20]:<\/div>\n
\n
\n
\n
<\/span>log_A<\/span> =<\/span> np<\/span>.<\/span>log10<\/span>(<\/span>A<\/span>)<\/span>\r\nprint<\/span> log_A<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
\n
<\/div>\n
\n
[[-4.          -inf -5.     ]\r\n [-3.      -2.30103      nan]]\r\n<\/pre>\n<\/div>\n<\/div>\n
\n
<\/div>\n
\n
C:\\Users\\Conrad\\Anaconda2\\lib\\site-packages\\ipykernel\\__main__.py:1: RuntimeWarning: divide by zero encountered in log10\r\n  if __name__ == '__main__':\r\nC:\\Users\\Conrad\\Anaconda2\\lib\\site-packages\\ipykernel\\__main__.py:1: RuntimeWarning: invalid value encountered in log10\r\n  if __name__ == '__main__':\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n<\/div>\n
\n
\n
This helps a bit, but unfortunately the ‘bad’ values put in a false color that says something it shouldn’t.<\/p>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
In [21]:<\/div>\n
\n
\n
\n
<\/span>plt<\/span>.<\/span>pcolormesh<\/span>(<\/span>log_A<\/span>,<\/span>vmin<\/span>=-<\/span>6<\/span>,<\/span>vmax<\/span>=-<\/span>3<\/span>)<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
\n
Out[21]:<\/div>\n
\n
<matplotlib.collections.QuadMesh at 0x870c4b0><\/pre>\n<\/div>\n<\/div>\n
\n
<\/div>\n
\n\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n<\/div>\n
\n
\n
A much better alternative is to mark the bad points as invalid.  Numpy’s masked array does this for us.  The correct invoccation is numpy.ma.masked_invalid, where ma is numpy’s masked array package and masked_invalid is the function that takes care of $-\\infty$ and $nan$.  The proper array is now<\/p>\n
$$ \\log_{10}(A) = \\left[ \\begin{array}{ccc}  -4 & — & -5 \\\\ -3 & -2.3 & —  \\end{array} \\right] $$\n<\/p><\/div>\n<\/div>\n<\/div>\n
\n
\n
In [22]:<\/div>\n
\n
\n
\n
<\/span>masked_log_A<\/span> =<\/span> np<\/span>.<\/span>ma<\/span>.<\/span>masked_invalid<\/span>(<\/span>log_A<\/span>)<\/span>\r\nprint<\/span> masked_log_A<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
\n
<\/div>\n
\n
[[-4.0 -- -5.0]\r\n [-3.0 -2.3010299956639813 --]]\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
In [23]:<\/div>\n
\n
\n
\n
<\/span>plt<\/span>.<\/span>pcolormesh<\/span>(<\/span>masked_log_A<\/span>,<\/span>vmin<\/span>=-<\/span>6<\/span>,<\/span>vmax<\/span>=-<\/span>3<\/span>)<\/span>\r\n<\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
\n
\n
\n
Out[23]:<\/div>\n
\n
<matplotlib.collections.QuadMesh at 0x875cc90><\/pre>\n<\/div>\n<\/div>\n
\n
<\/div>\n
\n\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
Define a typical scientific array¶ The definition of the matrix A is designed to represent a typical scientific data set that varies by orders of magnitude. In addition, some of the data is invalid due to either no measurement ($$A_{0,1} = 0$$) or there was a measurement glitch ($$A_{1,2} = -1$$). For sake of completeness, … Continue reading Masked Scientific Arrays<\/span> →<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-71","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/lop.blogwyrm.com\/index.php?rest_route=\/wp\/v2\/posts\/71","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lop.blogwyrm.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lop.blogwyrm.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lop.blogwyrm.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/lop.blogwyrm.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=71"}],"version-history":[{"count":2,"href":"https:\/\/lop.blogwyrm.com\/index.php?rest_route=\/wp\/v2\/posts\/71\/revisions"}],"predecessor-version":[{"id":73,"href":"https:\/\/lop.blogwyrm.com\/index.php?rest_route=\/wp\/v2\/posts\/71\/revisions\/73"}],"wp:attachment":[{"href":"https:\/\/lop.blogwyrm.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=71"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lop.blogwyrm.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=71"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lop.blogwyrm.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=71"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}