Python Number Conversion Chart

I ran across this Python number conversion chart a while ago on HackerNews. It looks pretty helpful – I always forget which way hexlify and unhexlify work!

Python Number Conversion Chart


Getting clever with list comprehensions

Suppose you have a list of TCP connections:

packets = [
    { 'src': ('192.168.1.1', 22045), 'dst': ('192.168.1.3', 31037) },
    { 'src': ('192.168.1.1', 11567), 'dst': ('192.168.1.2', 28432) },
    { 'src': ('192.168.1.1', 22045), 'dst': ('192.168.1.3', 31037) },
    { 'src': ('192.168.1.7', 1), 'dst': ('192.168.1.8', 2) }
]

If we just want to get a quick idea of who is talking to who (most data of this sort would be much larger than the example above), a quick count of how many packets are in each conversation is a good start. Here’s the simple code to parse that out:

packet_count = {}
for p in packets:
    if (p['src'], p['dst']) in packet_count:
        packet_count[(p['src'], p['dst'])] += 1
    else:
        packet_count[(p['src'], p['dst'])] = 1
print packet_count

--- output ---
{(('192.168.1.1', 11567), ('192.168.1.2', 28432)): 1,
 (('192.168.1.1', 22045), ('192.168.1.3', 31037)): 2,
 (('192.168.1.7', 1), ('192.168.1.8', 2)): 1}

But we can simplify this code a lot just by using defaultdict:

from collections import defaultdict
packet_count = defaultdict(int)
for p in packets:
        packet_count[(p['src'], p['dst'])] += 1
print packet_count

--- output ---
defaultdict(<type 'int'>, 
    {(('192.168.1.7', 1), ('192.168.1.8', 2)): 1, 
    (('192.168.1.1', 22045), ('192.168.1.3', 31037)): 2, 
    (('192.168.1.1', 11567), ('192.168.1.2', 28432)): 1})

I’ve got a while until this plane lands, so how about a clever one-liner?

packet_count = dict([((p['src'],p['dst']), packets.count(p)) 
                    for p in packets])
print packet_count

--- output ---
{(('192.168.1.1', 11567), ('192.168.1.2', 28432)): 1,
 (('192.168.1.1', 22045), ('192.168.1.3', 31037)): 2,
 (('192.168.1.7', 1), ('192.168.1.8', 2)): 1}

But wait, there’s more! Python 2.7 supports dictionary comprehensions, allowing us to do…

packet_count = {(p['src'],p['dst']) : packets.count(p) 
                    for p in packets}
print packet_count

--- output ---
{(('192.168.1.1', 11567), ('192.168.1.2', 28432)): 1,
 (('192.168.1.1', 22045), ('192.168.1.3', 31037)): 2,
 (('192.168.1.7', 1), ('192.168.1.8', 2)): 1}

Micro Book Review: Getting Started with D3

Getting Started with D3 by Mike Dewar reads like an online tutorial. If you fall into the very narrow niche of knowing what D3 is (because the acronym, Data-Driven Documents, is never defined in the book) and have no internet access and no computer, this book is for you. (As it happens, this was perfect for me).

The book is clear and concise, and I appreciated that the author used real-world data that is parsed and served with Python. Unfortunately, at just 70 pages the book feels unfinished. Getting Started with D3 is more cohesive and approachable than many online tutorials, but does not have much of an edge beyond that.


Automatic Warhol-Style Serigraph Generation

We recently had a logo drawn up for a research project that I’ve been involved with for a couple of years. I’ve been planning on having a poster-size version printed, but I thought that simply printing a very large logo would be too easy. I’m a fan of pop-art, so I thought that an Andy Warhol-style serigraph of our logo would make a cool poster.

My first thought was that this should be pretty easy in Photoshop. I hit the Internet to find tutorials for making Warhol-style graphics from an image and started Photoshop. I quickly abandoned that idea; I don’t know Photoshop well at all, and the entire process appeared painfully manual. So, naturally, I decided to do it programmatically with the Python Imaging Library.

I used Warhol’s Che Guevara serigraph as a model.


Original Andy Warhol Che Guevara Serigraph

I used the colors in the original by manually picked them out from the original piece and saving them in a list of dictionaries:

# colors here are taken directly from Warhol's Che Guevara serigraph
# (in order left to right, top to bottom)
colorset = [ 
    {'bg' : (255,255,0,255), 'fg' : (50,9,125,255), 'skin': (118,192,0,255)},
    {'bg' : (0,122,240,255), 'fg' : (255,0,112,255), 'skin': (255,255,0,255)},
    {'bg' : (50,0,130,255),'fg' : (255,0,0,255),'skin': (243,145,192,255)},
    {'bg' : (255,126,0,255),'fg' : (134,48,149,255),'skin': (111,185,248,255)},
    {'bg' : (255,0,0,255),'fg' : (35,35,35,255),'skin': (255,255,255,255)},
    {'bg' : (122,192,0,255),'fg' : (255,89,0,255),'skin': (250,255,160,255)},
    {'bg' : (0,114,100,255),'fg' : (252,0,116,255),'skin': (250,250,230,255)},
    {'bg' : (250,255,0,255),'fg' : (254,0,0,255),'skin': (139,198,46,255)},
    {'bg' : (253,0,118,255),'fg' : (51,2,126,255),'skin': (255,105,0,255)}
]

Automatically Warholifying images is not straightforward for general pictures or images, but working on vector-style images with a couple of a assumptions is not very difficult. The assumptions I made for this were that the “face” color is white and the background is transparent.

The image I’ll use for this example is a vector version of the classic photo of Che Guevara from Wikimedia Commons. I modified this image by using a bucket fill tool to make the face area white, but did not have to make any other modifications.


Vector image of Che Guevara

First, we change the transparent background to be the bg_color specified and change everything else to be fg_color.

def color_bg_fg(image, bg_color, fg_color):
    '''change transparent background to bg_color and change
    everything non-transparent to fg_color'''
    fg_layer = Image.new('RGBA', image.size, fg_color)
    bg_layer = Image.new('RGBA', image.size, bg_color) 
    masked_image = Image.composite(fg_layer, bg_layer, image)
    return masked_image

bg_fg_layer = color_bg_fg(image, bg_color, fg_color)

Output:


Background changed to bg_color and all foreground changed to fg_color

Next we create a mask of the skin area. The first step here is to change the background from transparent to something non-transparent (I chose black):

def darken_bg(image, color):
    '''composite image on top of a single-color image, effectively turning all
    transparent parts to that color'''
    color_layer = Image.new('RGBA', image.size, color) 
    masked_image = Image.composite(image, color_layer, image)
    return masked_image

temp_dark_image = darken_bg(image, (0,0,0,255))

Output:


Background changed to black

We then change all of the white-ish areas of the image to be transparent.

def white_to_color(image, color):
    '''change all colors close to white and non-transparent
    (alpha > 0) to be color.'''
    threshold=50
    dist=10
    arr=np.array(np.asarray(image))
    r,g,b,a=np.rollaxis(arr,axis=-1)    
    mask=((r>threshold)
          & (g>threshold)
          & (b>threshold)
          & (np.abs(r-g)<dist)
          & (np.abs(r-b)<dist)
          & (np.abs(g-b)<dist)
          & (a>0)
          )
    arr[mask]=color
    image=Image.fromarray(arr,mode='RGBA')
    return image

skin_mask = white_to_color(temp_dark_image,(0,0,0,0))

Output:


White 'skin' areas changed to transparent

We then create an image the same size as our source file that is nothing but skin_color.

skin_layer = Image.new('RGBA', image.size, skin_color) 

Output:


Single color image

Finally, we composite bg_gf_layer with skin_layer, using skin_mask as a mask.

out = Image.composite(bg_fg_layer, skin_layer, skin_mask)

Output:


Complete serigraph-style image

Here is all the steps put together:

def make_warhol_single(image, bg_color, fg_color, skin_color):
    '''create a single warhol-serigraph-style image'''
    bg_fg_layer = color_bg_fg(image, bg_color, fg_color)
    temp_dark_image = darken_bg(image, (0,0,0,255))
    skin_mask = white_to_color(temp_dark_image,(0,0,0,0))
    skin_layer = Image.new('RGBA', image.size, skin_color) 
    out = Image.composite(bg_fg_layer, skin_layer, skin_mask)
    return out

Finally, we can call the make_warhol_single function with multiple color combinations and create a single image containing all of them.

def warholify(image_file):
    im = Image.open(image_file).convert('RGBA')

    warhols = []
    for colors in colorset:
        bg = colors['bg']
        fg = colors['fg']
        skin = colors['skin']
        warhols.append(make_warhol_single(im, bg, fg, skin))

    x = im.size[0]
    y = im.size[1]

    blank_image = Image.new("RGB", (x*3, y*3))
    blank_image.paste(warhols[0], (0,0))
    blank_image.paste(warhols[1], (x,0))
    blank_image.paste(warhols[2], (x*2,0))
    blank_image.paste(warhols[3], (0,y))
    blank_image.paste(warhols[4], (x,y))
    blank_image.paste(warhols[5], (x*2,y))
    blank_image.paste(warhols[6], (0,y*2))
    blank_image.paste(warhols[7], (x,y*2))
    blank_image.paste(warhols[8], (x*2,y*2))

    blank_image.save('out.png')

Output:


Final image

The full source code for this program is available on GitHub.