Through the work with dogelang I am currently looking at pureness of python functions and on how to decide and annotate pureness so that I, in theory, could write a static checker that checks whether the annotated functions are pure.

We can take the definition of pureness from the wikipedia and it is as follows

1. The function always evaluates the same result value given the same argument value(s). The function result value cannot depend on any hidden information or state that may change while program execution proceeds or between different executions of the program, nor can it depend on any external input from I/O devices.
2. Evaluation of the result does not cause any semantically observable side effect or output, such as mutation of mutable objects or output to I/O devices.

Python makes few restrictions on side-effects and mutability and since we even can override basic operators (like the dot-operator for attribute access), it is hard to infer anything remotely useful or non-trivial from a python-program.

I have been, for example, responsible for code like that (paracoded for your convenience):

def cls(object):
  def __init__(self):
    self.d = {}

  def __getattr__(self, attr)
    if not attr in self.d:
      self.d[attr] = someobj(attr)
    return self.d[attr]

A = cls()
A.foo # this line has side effects

This violates the basic principle of python that an expression should either have a return value or a side-effect but not both. But we might have done this for good reason. In this case, the academic value of the "front-end" of the library outweighs the atrocities of the implementation. But it serves as a fine example that, it is hard to reason about even simple looking expressions, if they include the dot-operator (or any other operator for that matter), especially if you don't have the full class definition at hand.

But it isn't only user-made classes that don't hold true to that principle. Most low-level functions that interact with the operating system do that in some way. They have their side-effect within the realms of the rest of the system and still they have a return value that represents some aspect of the operation within our program.

os.write(fd, str)

Write the bytestring in str to file descriptor fd. Return the number of bytes actually written.

open(file, ...)

Open file and return a corresponding file object. If the file cannot be opened, an OSError is raised.

So we would want to mark any function as impure that has a reference to one of these impure functions. But we would also want to mark any functions defined within these impure functions to be themselves impure. Looking at the following code it would be hard to infere whether a function has a reference to, let's say 'open'.

o, f = open, "file.tmp"
def fun():
  o(f)

Given Rice's theorem, I'd think that properly tracking references would either be undecidable or at least a problem of exponential growth (tracking all possible assignments to some reference). So to be on the safe side, we state the following as our impurity rule number 1: A function is impure if it uses a reference from an impure function or has a reference to an impure function. We would also want an exhaustive list of builtin impure functions, applying the rules recursively.

In the former code example the surrounding block leaks (that's what we will call it) the reference to both o and f into the namespace of fun. This would not be a problem. We even want this behaviour. Let's look at this example:

def f():
  return x*x
x = 5
print(f()) # 25
x = 7
print(f()) # 49

This is contrary to our definition of purity, that the function, given the same arguments should evaluate to the same value. So this only happened, because the reference was updated. Let's add impurity rule number 2: A function is impure if a name is assigned twice and a reference to that name is leaked. This should cover all cases since shenanigans like the following cause a NameError.

def a():
  yield (lambda : x*x)
  x = 3
  yield None
x, i = 5, a()
g = next(i)
print(g()) # Name Error: free variable 'x' referenced before assignment in enclosing scope
next(i)
print(g())

Here we see the yield-keyword we have the case of suspended computation. The next-function is of course part of the set of impure builtin functions because it returns a value and advances the internal state of the generator. Since for rule 2 we introduced the leaked name, we can still allow multiple assignments within a function if the name is not leaked because we want to allow, for example, the generator-function infinity to be pure.

def infinity():
  i = 0
  while True:
    yield i
    i = i + 1

Here, I find, we arrive at an impasse. Every iteration over a generator "uses the generator up". We could say that we won't allow references to generator-objects but that seems arbitrarily strict. Also, we can't really know which name could be used by a generator-object which would lead to not allowing any bare names (only function calls). Disallowing generators seems impossible since map and filter and other functional primitives depend on it.

I would welcome any ideas for a rule that would solve this issue of leaking the internal state of a generator.

Previous: CoffeeScript is a missed opportunity , Next: talk at Leipzig's Haskell conference HaL-10