• About Me
  • Media Mentions
  • Contact

Social Metrics

Social Media, Organizations, and Academic Research

  • Email
  • Facebook
  • GitHub
  • LinkedIn
  • Pinterest
  • RSS
  • Twitter
  • Home
  • Blog
  • Publications
  • Teaching
  • Python
  • About Me
You are here: Home / python / Random Python Recipes

Random Python Recipes

April 26, 2015 by Gregory Saxton Leave a Comment

This page is mostly for me as a handy reference for all those Python commands I tend to forget. That said, if it proves helpful to any others, all the better!

Contents

  • 1 Lists
  • 2 Dictionaries
  • 3 Regular Expressions
  • 4 Miscellaneous

Lists

Create list by slicing items of another list:

b = [x[:8] for x in cusips_compustat]

Combine two lists into a dictionary:

dict(zip([1,2,3,4], [a,b,c,d]))

Get list of all files in a directory:

filenames = next(os.walk(directory_name))[2]

Find items in one list that are not in another:

a = [some list]
b = [another list]
c = []
for bx in b:
    if bx not in a:
        c.append(bx)
if c:
    print "these are the list elements of b not present in a:", c
else:
     print "no elements of list b are not in list a"
.

Dictionaries

Merge two dictionaries (same keys in both dictionaries):

d1 = nx.betweenness_centrality(G)
d2 = nx.degree_centrality(G)
finaldict = {key:(d1[key], d2[key]) for key in d1}

Same as above, but values as list instead of tuple:

finaldict = {key:[d1[key], d2[key]] for key in d1}

For converting to a PANDAS dataframe, you would normally want something like this — create a dictionary of dictionaries and add keys:

finaldict = {key:{'degree': d2[key], 'betweenness': d1[key]} for key in d2}

To convert the above dictionary to a dataframe:

df = pd.DataFrame.from_dict(finaldict, orient='index')

And you would want this if you are building a dictionary with more than one month:

finaldict = {key:{month:{'degree': D2[key], 'betweenness': d1[key]}} for key in d2}

To merge the above nested dictionary, you have to do something more complicated to convert to a DF:

ticks = []
frames = []
for i, d in finaldict.iteritems():
     ticks.append(i)
     frames.append(pd.DataFrame.from_dict(d, orient='index'))
df = pd.concat(frames, keys=ticks)

Sort dictionary by keys:

for key in sorted(mydict.iterkeys()):
    print "%s: %s" % (key, mydict[key])

Sort dictionary by numerical values:

for key, value in sorted(mydict.iteritems(), key=lambda (k,v): (v,k), reverse=True):
    print "%s: %s" % (key, value)
.

Regular Expressions

Find all instances of text in string starting with ‘Item ‘:

itemx = soup(text=re.compile('Item '))

Find all instances of text that start with ‘Item ‘ plus any number(s) then ‘.’ then any number(s):

match = re.search('Item (\d+).(\d+)', data)
#match = re.search('Item (\d+).(\d+)', data, re.IGNORECASE) #case-insensitive search
if match:
    item = match.group()
.

Miscellaneous

Loop over directory and find and open specific file:

indir = '/Users/BobSmith/Documents/SEC filings'
for root, dirs, filenames in os.walk(indir):
    for f in filenames:
        if f == '8-K.htm':
            file = indir+'/'+f
            data = open(os.path.join(root, f), 'r').read()

Rename a file:

import os
os.rename(dir_name+old_filename, dir_name+new_filename)
.
Share Button
image_pdfimage_print

Filed Under: python Tagged With: python

Recent Posts

  • Making a Contribution in Accounting Research, Part IV: Mapping the Conceptual Relationships in Nonprofit Accounting Articles
  • Making a Contribution in Accounting Research, Part II: Focus on Nonprofit Accounting
  • Making a Contribution in Accounting Research, Part I: Types of Contributions
  • Making a Contribution in Accounting Research, Part III: Relationships in Top Nonprofit Accounting Articles
  • Quest for Attention: Nonprofit Advocacy in a Social Media Age

Featured Posts

Python Data Analytics Tutorials

The bulk of my research involves some degree of 'Big Data' … [Read More...]

Downloading Tweets, Take III – MongoDB

In this tutorial I walk you through how to use Python and … [Read More...]

Does Twitter Matter?

Twitter is not the Gutenberg Press. The 'Big Data' … [Read More...]

Archives

  • November 2020
  • October 2020
  • July 2020
  • January 2019
  • October 2018
  • July 2018
  • May 2018
  • April 2018
  • March 2018
  • October 2017
  • September 2017
  • November 2016
  • October 2015
  • June 2015
  • May 2015
  • April 2015
  • November 2014
  • October 2014
  • September 2014
  • May 2014
  • April 2014

E-mail sign-up

Every time I post something new to my blog, receive it free by email.

No spam.

Contact Information

Gregory D. Saxton
Schulich School of Business
York University
Toronto, ON
gsaxton@yorku.ca

Recent Posts

  • Making a Contribution in Accounting Research, Part IV: Mapping the Conceptual Relationships in Nonprofit Accounting Articles
  • Making a Contribution in Accounting Research, Part II: Focus on Nonprofit Accounting
  • Making a Contribution in Accounting Research, Part I: Types of Contributions
  • Making a Contribution in Accounting Research, Part III: Relationships in Top Nonprofit Accounting Articles
  • Quest for Attention: Nonprofit Advocacy in a Social Media Age

Tag Cloud

academia academic research AccountingAnalytics arnova14 Big Data conference Data Analytics Database hashtags ica iPython MongoDB nonprofits PANDAS PhD_studies Programming python replication research social media socialmedia tutorial Twitter

Copyright © 2021 · Metro Pro Theme on Genesis Framework · WordPress · Log in