Random Python Recipes

This page is mostly for me as a handy reference for all those Python commands I tend to forget. That said, if it proves helpful to any others, all the better!

Lists

Create list by slicing items of another list:

b = [x[:8] for x in cusips_compustat]

Combine two lists into a dictionary:

dict(zip([1,2,3,4], [a,b,c,d]))

Get list of all files in a directory:

filenames = next(os.walk(directory_name))[2]

Find items in one list that are not in another:

a = [some list]<br />
b = [another list]<br />
c = []<br />
for bx in b:<br />
    if bx not in a:<br />
        c.append(bx)<br />
if c:<br />
    print &quot;these are the list elements of b not present in a:&quot;, c<br />
else:<br />
     print &quot;no elements of list b are not in list a&quot;

Dictionaries

Merge two dictionaries (same keys in both dictionaries):

d1 = nx.betweenness_centrality(G)<br />
d2 = nx.degree_centrality(G)<br />
finaldict = {key:(d1[key], d2[key]) for key in d1}

Same as above, but values as list instead of tuple:

finaldict = {key:[d1[key], d2[key]] for key in d1}

For converting to a PANDAS dataframe, you would normally want something like this — create a dictionary of dictionaries and add keys:

finaldict = {key:{'degree': d2[key], 'betweenness': d1[key]} for key in d2}

To convert the above dictionary to a dataframe:

df = pd.DataFrame.from_dict(finaldict, orient='index')

And you would want this if you are building a dictionary with more than one month:

finaldict = {key:{month:{'degree': D2[key], 'betweenness': d1[key]}} for key in d2}

To merge the above nested dictionary, you have to do something more complicated to convert to a DF:

<br />
ticks = []<br />
frames = []<br />
for i, d in finaldict.iteritems():<br />
     ticks.append(i)<br />
     frames.append(pd.DataFrame.from_dict(d, orient='index'))<br />
df = pd.concat(frames, keys=ticks)

Sort dictionary by keys:

<br />
for key in sorted(mydict.iterkeys()):<br />
    print &quot;%s: %s&quot; % (key, mydict[key])<br />

Sort dictionary by numerical values:

<br />
for key, value in sorted(mydict.iteritems(), key=lambda (k,v): (v,k), reverse=True):<br />
    print &quot;%s: %s&quot; % (key, value)<br />

Regular Expressions

Find all instances of text in string starting with ‘Item ‘:

itemx = soup(text=re.compile('Item '))

Find all instances of text that start with ‘Item ‘ plus any number(s) then ‘.’ then any number(s):

match = re.search('Item (\d+).(\d+)', data)<br />
#match = re.search('Item (\d+).(\d+)', data, re.IGNORECASE) #case-insensitive search<br />
if match:<br />
    item = match.group()<br />

Miscellaneous

Loop over directory and find and open specific file:

<br />
indir = '/Users/BobSmith/Documents/SEC filings'<br />
for root, dirs, filenames in os.walk(indir):<br />
    for f in filenames:<br />
        if f == '8-K.htm':<br />
            file = indir+'/'+f<br />
            data = open(os.path.join(root, f), 'r').read()<br />

Rename a file:

<br />
import os<br />
os.rename(dir_name+old_filename, dir_name+new_filename)<br />