This page is mostly for me as a handy reference for all those Python commands I tend to forget. That said, if it proves helpful to any others, all the better!
Lists
Create list by slicing items of another list:
[python]b = [x[:8] for x in cusips_compustat][/python]
Combine two lists into a dictionary:
[python]dict(zip([1,2,3,4], [a,b,c,d]))[/python]
Get list of all files in a directory:
[python]filenames = next(os.walk(directory_name))[2][/python]
Find items in one list that are not in another:
[python]a = [some list]
b = [another list]
c = []
for bx in b:
if bx not in a:
c.append(bx)
if c:
print "these are the list elements of b not present in a:", c
else:
print "no elements of list b are not in list a"[/python]
Dictionaries
Merge two dictionaries (same keys in both dictionaries):
[python]d1 = nx.betweenness_centrality(G)
d2 = nx.degree_centrality(G)
finaldict = {key:(d1[key], d2[key]) for key in d1}[/python]
Same as above, but values as list instead of tuple:
[code]finaldict = {key:[d1[key], d2[key]] for key in d1}[/code]
For converting to a PANDAS dataframe, you would normally want something like this — create a dictionary of dictionaries and add keys:
[python]finaldict = {key:{‘degree’: d2[key], ‘betweenness’: d1[key]} for key in d2}[/python]
To convert the above dictionary to a dataframe:
[python]df = pd.DataFrame.from_dict(finaldict, orient=’index’)[/python]
And you would want this if you are building a dictionary with more than one month:
[python]finaldict = {key:{month:{‘degree’: D2[key], ‘betweenness’: d1[key]}} for key in d2}[/python]
To merge the above nested dictionary, you have to do something more complicated to convert to a DF:
[python]
ticks = []
frames = []
for i, d in finaldict.iteritems():
ticks.append(i)
frames.append(pd.DataFrame.from_dict(d, orient=’index’))
df = pd.concat(frames, keys=ticks)[/python]
Sort dictionary by keys:
[python]
for key in sorted(mydict.iterkeys()):
print "%s: %s" % (key, mydict[key])
[/python]
Sort dictionary by numerical values:
[python]
for key, value in sorted(mydict.iteritems(), key=lambda (k,v): (v,k), reverse=True):
print "%s: %s" % (key, value)
[/python]
Regular Expressions
Find all instances of text in string starting with ‘Item ‘:
[python]itemx = soup(text=re.compile(‘Item ‘))[/python]
Find all instances of text that start with ‘Item ‘ plus any number(s) then ‘.’ then any number(s):
[python]match = re.search(‘Item (\d+).(\d+)’, data)
#match = re.search(‘Item (\d+).(\d+)’, data, re.IGNORECASE) #case-insensitive search
if match:
item = match.group()
[/python]
Miscellaneous
Loop over directory and find and open specific file:
[python]
indir = ‘/Users/BobSmith/Documents/SEC filings’
for root, dirs, filenames in os.walk(indir):
for f in filenames:
if f == ‘8-K.htm’:
file = indir+’/’+f
data = open(os.path.join(root, f), ‘r’).read()
[/python]
Rename a file:
[python]
import os
os.rename(dir_name+old_filename, dir_name+new_filename)
[/python]