Standard Library

What's new and interesting in Python Standard Library 2.7, 3.2 and 3.3
Senthil Kumaran
Python Core Developer

What is coming up new in Python 2.7, 3.2 and 3.3 versions

There are 3 versions of this presentation:

©2010, licensed under a Creative Commons Attribution/Share-Alike (BY-SA) license.

CPython

At the moment

./python -q
>>> import sys; sys.version
'3.3.0a1+'
...
./python3.2
'3.2.3rc1'
...
./python2.7
'3.2.3rc1'

faulthandler

This module contains functions to dump Python tracebacks explicitly, on a fault, after a timeout, or on a user signal. Call faulthandler.enable() to install fault handlers for the SIGSEGV, SIGFPE, SIGABRT, SIGBUS, and SIGILL signals.

More information available at

http://blog.python.org/2011/05/new-faulthandler-module-in-python-33.html

lzma

bz2 module

Behavior is similar to bunzip2 utility. Like gzip, you can concatenate two bzip2 files.

bzip2 -c /etc/passwd >/tmp/pass.bz2 bzip2 -c /etc/passwd >>/tmp/pass.bz2

bunzip2 will output both parts, generating two copies of the file.

So nothing needs to be done on compression, but uncompression needs to look for another chunk of compressed data after finishing one chunk.

os module

Some benchmarks -

http://code.google.com/p/pyftpdlib/issues/detail?id=152#c5

packaging

http://docs.python.org/dev/library/packaging.html#module-packaging

signal module

http://docs.python.org/dev/library/signal.html man sigprocmask man pthread_sigmask man sigwait man sigwaitinfo

socket module

ssl module

http://bugs.python.org/issue13634

sys module

>>> sys.thread_info
sys.thread_info(name='pthread', lock='semaphore', version='NPTL 2.13')

urllib package

>>> urlopen(Request('http://www.python.org', method='HEAD'))

urllib package

argparse - 3.2

import argparse

parser = argparse.ArgumentParser(
description = 'Manage servers', # main description for help epilog = 'Tested on Solaris and Linux') # displayed after help
parser.add_argument('action', # argument name
choices = ['deploy', 'start', 'stop'], # three allowed values help = 'action on each target') # help msg
parser.add_argument('targets',
metavar = 'HOSTNAME', # var name used in help msg nargs = '+', # require one or more targets help = 'url for target machines') # help msg explanation
parser.add_argument('-u', '--user', # -u or --user option
required = True, # make it a required argument help = 'login as user')

print(parser.parse_args('-h'.split()))

logging module - 3.2

with open('conf.json', 'r') as f:
        conf = json.load(f)
logging.config.dictConfig(conf)

from concurrent import futures - 3.2

The primary offering of the new module is a pair of executor classes for launching and managing calls. The goal of the executors is to make it easier to use existing tools for making parallel calls. They save the effort needed to setup a pool of resources, launch the calls, create a results queue, add time-out handling, and limit the total number of threads, processes, or remote procedure calls.

Ideally, each application should share a single executor across multiple components so that process and thread limits can be centrally managed. This solves the design challenge that arises when each component has its own competing strategy for resource management.

Both classes share a common interface with three methods: submit() for scheduling a callable and returning a Future object; map() for scheduling many asynchronous calls at a time, and shutdown() for freeing resources. The class is a context manager and can be used in a with statement to assure that resources are automatically released when currently pending futures are done executing.

functools - 3.2

>>> import functools
>>> @functools.lru_cache(maxsize=300)
>>> def get_phone_number(name):
        c = conn.cursor()
        c.execute('SELECT phonenumber FROM phonelist WHERE name=?', (name,))
        return c.fetchone()[0]
...
>>> get_phone_number(name)        # cached lookup

functools - 3.2

>>> get_phone_number.cache_info()
CacheInfo(hits=4805, misses=980, maxsize=300, currsize=300)
>>> get_phone_number = get_phone_number.__wrapped__    # uncached function

functools - 3.2

@total_ordering
class Student:
    def __eq__(self, other):
        return ((self.lastname.lower(), self.firstname.lower()) ==
                (other.lastname.lower(), other.firstname.lower()))
    def __lt__(self, other):
        return ((self.lastname.lower(), self.firstname.lower()) <
                (other.lastname.lower(), other.firstname.lower()))

itertools - 3.2

>>> from itertools import accumulate
>>> list(accumulate([8, 2, 50]))
[8, 10, 60]

collections

collections

>>> tally = Counter(dogs=5, cat=3)
>>> tally -= Counter(dogs=2, cats=8)    # saturating subtraction
>>> tally
Counter({'dogs': 3})

>>> tally = Counter(dogs=5, cats=3)
>>> tally.subtract(dogs=2, cats=8)      # regular subtraction
>>> tally
Counter({'dogs': 3, 'cats': -5})

unittest - 3.2

python -m unittest discover -s my_proj_dir -p _test.py
>>> TestCase().assertEqual(pow(2, 3), 8)

pyc directories - 3.2

WSGI 1.1.1

New string formatting 3.2

>>> import shelve
>>> d = shelve.open('tmp.shl')
>>> 'The {project_name} status is {status} as of {date}'.format_map(d)
'The testing project status is green as of February 15, 2011'

>>> class PlaceholderDict(dict):
        def __missing__(self, key):
            return '<{}>'.format(key)
>>> 'Hello {name}, welcome to {location}'.format_map(PlaceholderDict())
'Hello <name>, welcome to <location>'

threading 3.2

from threading import Barrier, Thread

def get_votes(site):
    ballots = conduct_election(site)
    all_polls_closed.wait()        # do not count until all polls are closed
    totals = summarize(ballots)
    publish(site, totals)

all_polls_closed = Barrier(len(sites))
for site in sites:
    Thread(target=get_votes, args=(site,)).start()

ast module

>>> from ast import literal_eval
>>> request = "{'req': 3, 'func': 'pow', 'args': (2, 0.5)}"
>>> literal_eval(request)
{'args': (2, 0.5), 'req': 3, 'func': 'pow'}

>>> request = "os.system('do something harmful')"
>>> literal_eval(request)
Traceback (most recent call last):
  ...
ValueError: malformed node or string: <_ast.Call object at 0x101739a10>

array module - 3.3

shutil - 3.3

Deprecation Warnings - 2.7

3.x Backported Features in 2.7

3.x Backported Features in 2.7

Dictionary Views

http://stackoverflow.com/questions/340850/python-3-0-dict-methods-return-views-why

Bug fixes in modules

There is more

print('{0} {1}'.format('Thank',' you!'))