Krzysztof Żuraw


Personal site

My thoughts on blogging

This week I decided to write something more about what are reasons I wrote this blog, what I am taking from it so if you want some more technical post please check other articles.

What I want from this blog

I started this blog as a reference to myself about things in software development that I learn along the way. Moreover, this blog is a place to present my own opinions but I really welcome any comments especially ones that expand my view.

Why blogging?

I see couple benefits of blogging. First of all, you are learning. And I mean a lot- every blog post I write teach me something- either how to format my code in code snippets or implementation specific things. People who giving comments either via reddit or disqus teach me something. And I really mean that. Before I started writing content I knew that there will be people who are just haters and while I was thinking how to be invulnerable to such persons I came to the conclusion that every guy who left the comment on my blog wants to teach me something- I try not to take things personally.

Another thing is also connected with learning- your are learning not even how to code but you also learn how to write, to write in English. This language isn't my native one so I still have trouble with writing long passages of text in English but with every post, I make myself better.

The last thing with learning is my belief that the best way to learn something is to teach this thing to another human being. You have to know a big deal of subject if you want to write a good blog post and during writing, you are thinking how to present this topic in simple and consistent manner.

Conclusion and why I write this post

I made this quick blog post for the remainder to myself why I made this blog as well as for readers who may be interested in figuring out for what reason I write. But the most important thing is: I made this post because I don't have time this week to prepare proper blog post and I added some features to ease sharing content wrote by me.

Cover image by StockSnap under CC0.

To see comments and full article enter: My thoughts on blogging

Factory pattern in python

What is factory pattern all about? It introduces abstraction. In other words: helps software developers with the problem of creating objects without knowing exact class of the object that will be created.

Why use it? From programmers stack exchange:

(...) they allow the project to follow the SOLID principles more closely. In particular, the interface segregation and dependency inversion principles.

Factories and interfaces allow for a lot more long term flexibility. It allows for a more decoupled - and therefore more testable - design. (...)

  • It allows you to introduce an IoC container easily
  • It makes your code more testable as you can mock interfaces
  • It gives you a lot more flexibility when it comes time to change the application (i.e. you can create new implementations without changing the dependent code)

As we know why to use it let's jump into the code:

class BaseArchive(object):
    EXTENSION = None

    def __init__(self, location_path, files_to_pack):
        self.location_path = location_path
        self.files_to_pack = files_to_pack

    def generate(self):
        raise NotImplementedError()

At first, I created a simple base class to have a common interface for all archive classes. There is nothing sophisticated here: only setting up all necessary arguments in __init__ and telling that all children of BaseArchive have to implement generate method. Let's look how these children look like:

from zipfile import ZipFile
import tarfile


class ZIPArchive(BaseArchive):

    def generate(self):
        with ZipFile(self.location_path, 'w') as zip_file:
            for file_ in self.files_to_pack:
                zip_file.write(file_)


class TARArchive(BaseArchive):

    def generate(self):
        with tarfile.open(self.location_path, 'w') as tar_file:
            for file_ in self.files_to_pack:
                tar_file.add(file_)

Basically, ZIPArchive and TARArchive classes have generate method to create archives in given format using ZipFile or tarfile modules from standardlib as follows:

zip_archive = ZIPArchive(os.path.join(os.getcwd(), 'zip.zip'), ['for_zip'])
zip_archive.generate()
tar_archive = TARArchive(os.path.join(os.getcwd(), 'tar.tar'), ['for_tar.txt'])
tar_archive.generate()

Then the real fun begins. Say I want to generate archive just only by providing path and files to pack into the archive. For this purpose I use something called ArchiveManager which is factory:

class ArchiveManager(object):
    ARCHIVE_ENGINES = [ZIPArchive, TARArchive]

    def __init__(self, location_path, files_to_pack):
        self.location_path, self.extension = os.path.splitext(location_path)
        self.files_to_pack = files_to_pack
        self.archive_engine = self.choose_archive_engine()

    def choose_archive_engine(self):
        for engine in self.ARCHIVE_ENGINES:
            if engine.check_extenstion(self.extension):
                return engine(self.location_path, self.files_to_pack)

    def create_archive(self):
        self.archive_engine.generate()

Here I have ARCHIVE_ENGINES class attribute for every archive engine that I want to use. Below in __init__ I setup all necessary attributes with extension to use in choose_archive_engine. Here is the place where factory pattern starts to work: I iterate through all engines to check_extenstion and if there is a match I simply return the correct engine to self.archive_engine. To use that I need to write some more code in BaseArchive:

class BaseArchive(object):

  # rest of the code

  @classmethod
  def check_extenstion(cls,extension):
      return extension == cls.EXTENSION:

This check_extenstion that is classmethod helps me in figuring out which underlying archive class should I use. Thanks to that in ArchiveManager.create_archive I only need to provide self.archive_engine.generate(). This method doesnt't know if there is ZIPArchive or TARArchive class being used.

This was quick introduction how to use factory pattern in python. The code for this is available in this repo. Do you know more usages? Or maybe you don't agree with what I write- feel free to comment.

Edits (12.08.2016):

  • Refactor of check_extenstion method

Cover image by Tpsdave released into public domain.

To see comments and full article enter: Factory pattern in python

Design by contract in python- part three

I covered py.contracts and zope.interface, now it's time to write about abc module from python standard library.

I wanted to write about another library dpcontrancts, but unfortunately, I wasn't able to download it from PyPi.

When I was reading and reviewing material for previous posts I found out that there is a way to use python standard library abc.ABCMeta for contracts- I decided to give a try.

First, you have to know what are metaclasses: understanding python metaclasses and what are they needed for: Why use Abstract Base Classes in Python?.

After this introduction let's jump straight to the code:

import abc

class ExternalAPIPortMetaclass(object, metaclass=abc.ABCMeta):

    @abc.abstractmethod
    def search(self, query, *args, **kwargs):
        if not isinstance(query, str):
            raise ValueError('Query should be string')


class ExternalAPIPort(ExternalAPIPortMetaclass):

    def __init__(self, adapter):
        self.adapter = adapter

    def search(self, query, *args, **kwargs):
        super(ExternalAPIPort, self).search(query, *args, **kwargs)
        return self.adapter.search(query, *args, **kwargs)

What is happening here? I defined ExternalAPIPortMetaclass as a metaclass that inherits from abc.ABCMeta (This code snippet is valid for python 3). Then I decided to make abstractmethod called search so all instances of that metaclass will have to provide such function. Inside this code, I check whether provided query is a string or not. In ExternalAPIPort which inherits from previously defined I have to call super for ExternalAPIPortMetaclass search method. Thanks to that I can make a validation of query. Right after that I simply return search query.

What I don't like there is fact that I need to add additional line of code inside ExternalAPIPort.search with super just for checking contract which can trick others. That's why I think that metaclasses and contracts are two different topic besides that they have some pieces in common (both are designed for telling: here I make contract that you must obey).

To sum up this whole series I belive contracts are usefull for telling others that I made agrement that this function has to take and return certain value. In python word where is so called ducktyping and I don't think they are necessary in every case but designing by contracts can be helpfull as I shown in examples. In my day to day work I use contracts the same way to make agrements on ports methods in ports and adapters design pattern.

Thank you for reading! Feel free to comment!

Other blog posts in this series:

Edits (01.08.2016):

  • Add link to other posts with same series (thanks to alb1)

Cover image under CC0.

To see comments and full article enter: Design by contract in python- part three

Design by contract in python- part two

Last time I wrote about py.contracts- today is the time for another python library that helps to design by contracts.

This library is called zope.interface. From its GitHub page:

Interfaces are a mechanism for labeling objects as conforming to a given API or contract. So, this package can be considered as implementation of the Design By Contract methodology support in Python.

How is it translating to my example of ExternalAPIPort from the last post? First I have to create interface for my port:

from zope.interface import Interface, implements

class IExternalAPIPort(Interface):
    """A port will perform search in external service."""

    def search():
        """Perform search in external service."""

class ExternalAPIPort(object):
    implements(IExternalAPIPort)

    def __init__(self, adapter):
        self.adapter = adapter

    def search(self, query, *args, **kwargs):
        return self.adapter.search(query, *args, **kwargs)

Running this in python 3 will give you following traceback:

File "reddit-stars/external_api/external_api_port.py", line 11, in <module>
  class ExternalAPIPort(object):
File "reddit-stars/external_api/external_api_port.py", line 12, in ExternalAPIPort
  implements(IExternalAPIPort)
File ".virtualenvs/reddit-stars/lib/python3.5/site-packages/zope/interface/declarations.py", line 412, in implements
  raise TypeError(_ADVICE_ERROR % 'implementer')
TypeError: Class advice impossible in Python3.  Use the @implementer class decorator instead.

So to use it with python 3 you have to:

from zope.interface import Interface, implementer

class IExternalAPIPort(Interface):

    def search():
        pass

@implementer(IExternalAPIPort)
class ExternalAPIPort(object):

    def __init__(self, adapter):
        self.adapter = adapter

    def search(self, query, *args, **kwargs):
        return self.adapter.search(query, *args, **kwargs)

Thanks to that I have access to some checks that can help me with testing:

>>> IExternalAPIPort.implementedBy(ExternalAPIPort)
True
>>> IExternalAPIPort.providedBy(instantiated_port)
True

But how to write a contract? It's simple- let's start by creating invariant:

def search_invariant(obj):
    if not obj:
      raise ValueError('query cannot be empty')
    elif not isinstance(obj, str):
      raise ValueError('query has to be string')

class IExternalAPIPort(Interface):
    adapter = Attribute('external service')

def search():
    pass

invariant(search_invariant)

Right now when you run it, you will receive following error that query either has to be a string or not be empty:

File "/Development/reddit-stars/reddit_stars/urls.py", line 19, in <module>
   from search.views import RedditSearchView
 File "/Development/reddit-stars/search/views.py", line 4, in <module>
   from .forms import RedditSearchForm
 File "/Development/reddit-stars/search/forms.py", line 3, in <module>
   from external_api.external_api_port import instantiated_port #, fake_port
 File "/Development/reddit-stars/external_api/external_api_port.py", line 30, in <module>
   IExternalAPIPort.validateInvariants(1)
 File "/.virtualenvs/reddit-stars/lib/python3.5/site-packages/zope/interface/interface.py", line 438, in validateInvariants
   call(obj)
 File "/Development/reddit-stars/external_api/external_api_port.py", line 8, in search_invariant
   raise ValueError('query has to be string')
ValueError: query has to be string
# or
ValueError: query cannot be empty

Contract works! It assures that in running time that query will have to have values that are specified in a contract.

You can even declare your own invariant errors like this:

from zope.interface import Invalid

class QueryError(Invalid):
    def __str__(self):
        return "QueryError({})".format(*self.args)

def search_invariant(obj):
    if not obj:
        raise QueryError('query cannot be empty')
    elif not isinstance(obj, str):
        raise QueryError('query has to be string')

And right now traceback will show custom error:

external_api.external_api_port.QueryError: QueryError(query cannot be empty)

Unfortunately, I couldn't find the way to declare contract for return value using zope.intefrace. If you found one please let me know.

That's all for today! I hope you find this blog post interesting- feel free to comment. In the next blog post, I'll write about another library in python for contracts and some tools from the standard library.

Other blog posts in this series:

Edits (01.08.2016):

  • Add link to other posts with same series (thanks to alb1)

Cover image under CC0.

To see comments and full article enter: Design by contract in python- part two

Design by contract in python- part one

What exactly is design by contract? What is good and what is bad in this approach? What are python libraries that provide support for such design? You can find all these answers in this blog series about contracts.

Recently I read a Pragmatic Programmer. In this book there is a chapter 21: Design by Contract that stays:

It is a simple yet powerful technique that focuses on documenting (and agreeing to) the rights and responibilites of software mdoules to ensure program correctness.

How to achieve this? There are 3 basic expectations of any given function/class:

  1. Preconditions: What are function requirements in order to be called
  2. Postconditions: What is class guaranteed to do
  3. Class invariants: Conditions that are true after execution of function/class

Let's move to the actual implementation in python. I found at least 3 libraries that are made to provide help while using contracts. I'll start from the first one: py.contracts.

To demonstrate how to use this library I'll reuse my old code from ports and apdaters series. There is a reddit port:

class ExternalAPIPort(object):

    def __init__(self, adapter):
        self.adapter = adapter

    def search(self, query, *args, **kwargs):
        return self.adapter.search(query, *args, **kwargs)

I want to make sure that query is a string. What is more, I want this string not to be empty. And I return search should return nested list. How do I contract it using py.contracts?

from contracts import contract

# class ExternalAPIPort here

@contract(query='str[>0]', returns='list(str)')
def search(self, query, *args, **kwargs):

And messing around with wrong returns values gives you following errors:

contracts.interface.ContractNotRespected: Breach for return value of ExternalAPIPort:search().
Expected a list, got 'int'.
checking: list(str)   for value: Instance of <class 'int'>: 1

Right now I knew because of contract that this search method will take query parameter which has to be string with length more than 0 (precondition) and returns list of strings (postcondition).

The same can be accomplished with metaclasses:

from contracts import contract, ContractsMeta

class BasePort(object):
    __metaclass__ = ContractsMeta

    @abstractmethod
    @contract(query='str[>0]', returns='list(str)')
    def search(self, query, *args, **kwargs):
        pass

class ExternalAPIPort(BasePort):

  def search(self, query, *args, **kwargs):
    return self.adapter.search(query, *args, **kwargs)

This code will work for python 2. If you want to use ContractsMeta with python 3 you have to use function with_metaclass or simply write BasePort(object, metaclass=ContractsMeta):

from contracts import contract, ContractsMeta, with_metaclass

  class BasePort(with_metaclass(ContractsMeta, object)):

    @abstractmethod
    @contract(query='str[>0]', returns='list(str)')
    def search(self, query, *args, **kwargs):
        pass

What I like in this library is a possibility to disable contracts by calling contracts.disable_all() or using DISABLE_CONTRACTS environmental variable.

Unfortunately, py.contracts doesn't provide way to use invariants but you always can use assert.

That's all for this blog post! Feel free to comment and in next week I will look into another contract library in python.

Other blog posts in this series:

Edits (01.08.2016):

  • Add additional method for invoking metaclass in py.contracts (thanks to mm_ma_ma)
  • Add link to other posts with same series (thanks to alb1)

Cover image under CC0.

To see comments and full article enter: Design by contract in python- part one


Page 1 / 6