All posts by BW_admin
File Server Scraping
Within Python the best way to do file server scraping (or dumpster diving) is something like
def find_OpPt_CDFs(obs,ver,targ_dir):
file_pattern = re.compile('%s_d[ei]s\d_engr_l1_sigthrsh_\d{14}_v%s.cdf' % (obs,ver))
print 'scraping in %s' % targ_dir
filtered_files = []
for root, dirs, files in os.walk(targ_dir):
for file in files:
if file_pattern.match(file):
full_filename = (root+'/'+file).replace('\\','/')
filtered_files.append(full_filename)
return np.array(filtered_files).flatten()
Note that the regex compile step uses:
[ei] - either 'e' or 'i'
\d - digit
\d{14} - any 14 digits in a row
Color schemes for the Columns
The color scheme for the highlight boxes for each of the columns are:
Under the Hood: <div style="background-color: #d5d5dc; border: solid 1px black;">
Common Cents: <div style="background-color: #f5f5dc; border: solid 1px black;">
Aristotle to Digital: <div style = "background-color:#ff6666; border: solid 1px black;">
About Comics: <div style="background-color: #dee7d2;border: solid 1px black">