maweki's tech-nicalities

Giving Change with PostgreSQL

2024-04-25T00:00:00+02:00

For one of my courses I wanted to create a task to learn about problem solving techniques using recursive common table expressions in PostgreSQL. So we want to make change change (giving back money from the till after a purchase). As a general algorithm we will do the following:

to make change for $X$: Give out coin/bill for $Y$ with $Y \leq X$ and make change for $X-Y$ until $X=Y$

First we create our denominations that we can give out. We will leave out the 1ct coin. This will be example later on, because this makes greedy algorithms (always give out the largest possible denominator) fail. If you want to give out 6ct and choose to give out 5ct first, then you can't make change for the remaining cent. So by leaving out the 1ct coin, we have to keep this in mind from the beginning.

CREATE TABLE denoms ( v NUMERIC(5,2) PRIMARY KEY );
INSERT INTO denoms VALUES
(100), (50), (20), (10), (5), (2), (1), (0.5), (0.2), (0.1), (0.05), (0.02);

Let us try out to give 17ct out first. Our initial recursive call will have the money we still have to give out, and a list (text) of denominations we already have given out. And then we just give out any denominations and print the one where all money is given out:

WITH RECURSIVE r AS (
  SELECT 0.17 remaining, '' given
UNION
  SELECT r.remaining - denoms.v, given || ', ' || denoms.v
  FROM r JOIN denoms ON (denoms.v <= r.remaining)
)
SELECT given FROM r WHERE remaining = 0;

| given                                      |
| ------------------------------------------ |
| , 0.02, 0.05, 0.10                         |
| , 0.02, 0.10, 0.05                         |
| , 0.05, 0.02, 0.10                         |
| , 0.05, 0.10, 0.02                         |
| , 0.10, 0.02, 0.05                         |
| , 0.10, 0.05, 0.02                         |
| , 0.02, 0.05, 0.05, 0.05                   |
| , 0.05, 0.02, 0.05, 0.05                   |
| , 0.05, 0.05, 0.02, 0.05                   |
| , 0.05, 0.05, 0.05, 0.02                   |
| , 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.05 |
| , 0.02, 0.02, 0.02, 0.02, 0.02, 0.05, 0.02 |
| , 0.02, 0.02, 0.02, 0.02, 0.05, 0.02, 0.02 |
| , 0.02, 0.02, 0.02, 0.05, 0.02, 0.02, 0.02 |
| , 0.02, 0.02, 0.05, 0.02, 0.02, 0.02, 0.02 |
| , 0.02, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02 |
| , 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02 |

The main thing to notice here is, that we have each solution multiple times, as there is basically no order to us giving out the coins. Let's add to our algorithm the following: Once we have given out a denomination, we no longer give out larger denominations. This changes our recursive table to also include the largest denomination there is, and the join needs to take that into account:

WITH RECURSIVE r AS (
  SELECT 0.17 remaining, '' given, (SELECT max(v) FROM denoms) largest
UNION
  SELECT r.remaining - denoms.v, given || ', ' || denoms.v, denoms.v
  FROM r JOIN denoms ON (denoms.v <= r.remaining AND denoms.v <= r.largest)
)
SELECT given FROM r WHERE remaining = 0;

| given                                      |
| ------------------------------------------ |
| , 0.10, 0.05, 0.02                         |
| , 0.05, 0.05, 0.05, 0.02                   |
| , 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02 |

This is now much nicer and faster. But the number of variants grows quite fast and with 86ct we already have over a hundred possible ways to make change. I mean, this is correct but overall not very useful.

So let's leave the realm of completeness and add to our algorithm, that we only ever try the two largest possible denominations and not all of them. Ideally, we would be using a subqery instead of the full denoms table, but we are not allowed to access r in the subqery. So something like this would not be allowed:

WITH RECURSIVE r AS (
  SELECT 0.86 remaining, '' given, (SELECT max(v) FROM denoms) largest
UNION
  SELECT r.remaining - denoms.v, given || ', ' || denoms.v, denoms.v
  FROM r, (SELECT v FROM denoms WHERE denoms.v <= r.remaining AND denoms.v <= r.largest ORDER BY v DESC LIMIT 2) denoms
  -- this does not work!
)
SELECT given FROM r WHERE remaining = 0;

Instead, we can write all the stuff under WHERE, filtering out exactly which kinds of denominations we actually want to give out:

WITH RECURSIVE r AS (
  SELECT 0.86 remaining, '' given, (SELECT max(v) FROM denoms) largest
UNION
  SELECT r.remaining - denoms.v, given || ', ' || denoms.v, denoms.v
  FROM r, denoms WHERE denoms.v in (SELECT v FROM denoms WHERE v <= r.remaining AND denoms.v <= r.largest ORDER BY V DESC LIMIT 2)
)
SELECT given FROM r WHERE remaining = 0;

| given                                                                                                                                                              |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| , 0.50, 0.20, 0.10, 0.02, 0.02, 0.02                                                                                                                               |
| , 0.50, 0.20, 0.05, 0.05, 0.02, 0.02, 0.02                                                                                                                         |
| , 0.50, 0.10, 0.10, 0.10, 0.02, 0.02, 0.02                                                                                                                         |
| , 0.20, 0.20, 0.20, 0.20, 0.02, 0.02, 0.02                                                                                                                         |
| , 0.50, 0.10, 0.10, 0.05, 0.05, 0.02, 0.02, 0.02                                                                                                                   |
| , 0.20, 0.20, 0.20, 0.10, 0.10, 0.02, 0.02, 0.02                                                                                                                   |
| , 0.50, 0.10, 0.05, 0.05, 0.05, 0.05, 0.02, 0.02, 0.02                                                                                                             |
| , 0.20, 0.20, 0.20, 0.10, 0.05, 0.05, 0.02, 0.02, 0.02                                                                                                             |
| , 0.20, 0.20, 0.10, 0.10, 0.10, 0.10, 0.02, 0.02, 0.02                                                                                                             |
| , 0.20, 0.20, 0.10, 0.10, 0.10, 0.05, 0.05, 0.02, 0.02, 0.02                                                                                                       |
| , 0.20, 0.10, 0.10, 0.10, 0.10, 0.10, 0.10, 0.02, 0.02, 0.02                                                                                                       |
| , 0.20, 0.20, 0.10, 0.10, 0.05, 0.05, 0.05, 0.05, 0.02, 0.02, 0.02                                                                                                 |
| , 0.20, 0.10, 0.10, 0.10, 0.10, 0.10, 0.05, 0.05, 0.02, 0.02, 0.02                                                                                                 |
| , 0.50, 0.10, 0.05, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02                                                                                           |
| , 0.20, 0.20, 0.10, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.02, 0.02, 0.02                                                                                           |
| , 0.20, 0.10, 0.10, 0.10, 0.10, 0.05, 0.05, 0.05, 0.05, 0.02, 0.02, 0.02                                                                                           |
| , 0.20, 0.10, 0.10, 0.10, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.02, 0.02, 0.02                                                                                     |
| , 0.20, 0.20, 0.10, 0.10, 0.05, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02                                                                               |
| , 0.20, 0.10, 0.10, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.02, 0.02, 0.02                                                                               |
| , 0.20, 0.20, 0.10, 0.05, 0.05, 0.05, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02                                                                         |
| , 0.20, 0.10, 0.10, 0.10, 0.10, 0.05, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02                                                                         |
| , 0.20, 0.10, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.02, 0.02, 0.02                                                                         |
| , 0.20, 0.10, 0.10, 0.10, 0.05, 0.05, 0.05, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02                                                                   |
| , 0.20, 0.10, 0.10, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02                                                             |
| , 0.20, 0.20, 0.10, 0.05, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02                                                       |
| , 0.20, 0.10, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02                                                       |
| , 0.20, 0.10, 0.10, 0.10, 0.05, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02                                                 |
| , 0.20, 0.10, 0.10, 0.05, 0.05, 0.05, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02                                           |
| , 0.20, 0.10, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02                                     |
| , 0.20, 0.10, 0.10, 0.05, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02                         |
| , 0.20, 0.10, 0.05, 0.05, 0.05, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02                   |
| , 0.20, 0.10, 0.05, 0.05, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02, 0.02 |

Okay, this is still a lot. What we can alternatively do, is prevent taking a denomination that would need more than 5 of it for the remainder. That exclude the not-so-nice solutions:

WITH RECURSIVE r AS (
  SELECT 0.86 remaining, '' given, (SELECT max(v) FROM denoms) largest
UNION
  SELECT r.remaining - denoms.v, given || ', ' || denoms.v, denoms.v
  FROM r JOIN denoms ON (denoms.v <= r.remaining AND denoms.v <= r.largest AND denoms.v*5 >= r.remaining)
)
SELECT given FROM r WHERE remaining = 0;

| given                                                        |
| ------------------------------------------------------------ |
| , 0.50, 0.20, 0.10, 0.02, 0.02, 0.02                         |
| , 0.20, 0.20, 0.20, 0.20, 0.02, 0.02, 0.02                   |
| , 0.50, 0.10, 0.10, 0.10, 0.02, 0.02, 0.02                   |
| , 0.50, 0.20, 0.05, 0.05, 0.02, 0.02, 0.02                   |
| , 0.20, 0.20, 0.20, 0.10, 0.10, 0.02, 0.02, 0.02             |
| , 0.50, 0.10, 0.10, 0.05, 0.05, 0.02, 0.02, 0.02             |
| , 0.20, 0.20, 0.10, 0.10, 0.10, 0.10, 0.02, 0.02, 0.02       |
| , 0.20, 0.20, 0.20, 0.10, 0.05, 0.05, 0.02, 0.02, 0.02       |
| , 0.20, 0.20, 0.10, 0.10, 0.10, 0.05, 0.05, 0.02, 0.02, 0.02 |

And there we have it. That seems nice. Let's try out just a final larger example, with smaller maximum cardinality (3), just to check.

WITH RECURSIVE r AS (
  SELECT 35.99 remaining, '' given, (SELECT max(v) FROM denoms) largest
UNION
  SELECT r.remaining - denoms.v, given || ', ' || denoms.v, denoms.v
  FROM r JOIN denoms ON (denoms.v <= r.remaining AND denoms.v <= r.largest AND denoms.v*3 >= r.remaining)
)
SELECT given FROM r WHERE remaining = 0;

| given                                                                      |
| -------------------------------------------------------------------------- |
| , 20.00, 10.00, 5.00, 0.50, 0.20, 0.20, 0.05, 0.02, 0.02                   |
| , 20.00, 10.00, 5.00, 0.50, 0.20, 0.10, 0.10, 0.05, 0.02, 0.02             |
| , 20.00, 10.00, 2.00, 2.00, 1.00, 0.50, 0.20, 0.20, 0.05, 0.02, 0.02       |
| , 20.00, 10.00, 2.00, 2.00, 1.00, 0.50, 0.20, 0.10, 0.10, 0.05, 0.02, 0.02 |

That seems nice.

alternatively, we can also additionally count the number of coins/bills given out and try to use the fewest:

WITH RECURSIVE r AS (
  SELECT 119.99 remaining, '' given, (SELECT max(v) FROM denoms) largest, 0 cnt
UNION
  SELECT r.remaining - denoms.v, given || ', ' || denoms.v, denoms.v, r.cnt + 1
  FROM r JOIN denoms ON (denoms.v <= r.remaining AND denoms.v <= r.largest AND denoms.v*5 >= r.remaining)
)
SELECT given, cnt FROM r WHERE remaining = 0 ORDER BY cnt ASC LIMIT 5;

| given                                                                       | cnt |
| --------------------------------------------------------------------------- | --- |
| , 100.00, 10.00, 5.00, 2.00, 2.00, 0.50, 0.20, 0.20, 0.05, 0.02, 0.02       | 11  |
| , 100.00, 5.00, 5.00, 5.00, 2.00, 2.00, 0.50, 0.20, 0.20, 0.05, 0.02, 0.02  | 12  |
| , 100.00, 10.00, 5.00, 2.00, 1.00, 1.00, 0.50, 0.20, 0.20, 0.05, 0.02, 0.02 | 12  |
| , 100.00, 10.00, 5.00, 2.00, 2.00, 0.50, 0.20, 0.10, 0.10, 0.05, 0.02, 0.02 | 12  |
| , 50.00, 50.00, 10.00, 5.00, 2.00, 2.00, 0.50, 0.20, 0.20, 0.05, 0.02, 0.02 | 12  |

And there we go, this is how we make change in PostgreSQL using the RECURSIVE keyword (it's actually more of a co-recursive thing, if you ask me) in Common Table Expressions.

How Many Boxes are on the Trailer?

2024-03-07T00:00:00+01:00

I recently came across an interesting puzzle:

Without shading or a 3D-view, we can not give a definitive answer. Taking a look at the side view, we can give an upper bound. If we expect, from the side view, every visible cube to have two invisible boxes behind it, we get to 51 cubes. But what is the lower bound?

I thought, it would be a good idea and exercise, to solve this problem using a constraint solver. First I needed to decide on a model. As there exists a grid of $7*3*3$ fields where each contains either a box or it does not. That sounds suspiciously like something Boolean. So let's model it as a Boolean satisfiability problem (SAT).

Since I haven't before, I wanted to try out PySAT and take it for a spin: The SAT solvers work with Boolean variables without names, just numbers. Additionally, our interface will create additional variables when we create more complex formulas due to Tseytin transformation. Tseytin transformation essentially creates a variable representing a subformula and the constraints for equivalence between the valuation of the newly created variable and the subformula. If you now want to create a new formula that contains the original formula as a subformula, you can refer to the single newly created variable, representing the whole subformula.

Because, as I said, the interface with the solver will be numbers-based, we need to decide on some representation or indexing between the solver's variables and our domain. We'll use a namedtuple for a Position P on the trailer, where x is along the depth of the trailer when viewed from the back, y is along the width of the trailer when viewed from the back, and z is along the height of the trailer, with the origin being at the bottom front left. We also need an empty list of constraints that we add to along the way:

from pysat.formula import *
from collections import namedtuple

P = namedtuple('P', ['x', 'y', 'z'])

boxes = {}

depth = 7
width = 3
height = 3

for x in range(depth):
  for y in range(width):
    for z in range(height):
      boxes[(x,y,z)] = Atom(P(x,y,z))

constraints = []

Instead of a namedtuple i initially wanted to use anonymous 3-tuples, but a bug in the library prevented me from doing that.

First we want to create the constraints for the side view. If we can see a box from that side, we know that at least one space in that group of 3 is filled by a box. We can simply use a disjunction over those three variables. If the space is empty, we know that there are no boxes. We can negate the disjunction. We "draw" the shape to make sure we got it right:

# side view
X = True
O = False
side_view = [
  [X,X,X,X,O,O,O],
  [X,X,X,X,X,X,O],
  [X,X,X,X,X,X,X],
]
for x in range(depth):
  for z in range(height):
    if side_view[-z-1][x]:
      constraints.append(Or(*[boxes[(x,y,z)] for y in range(width)]))
    else:
      constraints.append(Neg(Or(*[boxes[(x,y,z)] for y in range(width)])))

For the top view and back view we do not need to draw the shape that we are seeing, because every row/column/stack has a box in it and we do not need to consider the other case.

# back view
for y in range(width):
  for z in range(height):
    constraints.append(Or(*[boxes[(x,y,z)] for x in range(depth)]))

# top view
for y in range(width):
  for x in range(depth):
    constraints.append(Or(*[boxes[(x,y,z)] for z in range(height)]))

This looks like we are finished, but we have not yet considered gravity. If a box is anywhere not on the bottom level, it has to have another box under it. That is an implication:

# gravity
for x in range(depth):
  for y in range(width):
    for z in range(height-1):
      constraints.append(Implies(boxes[(x,y,z+1)], boxes[(x,y,z)]))

When I tried this out initially, there was a bug in the pySAT library where you could not use implications that are not on the top-level. This has been fixed quite fast.

Now we want to know the minimum number of needed boxes. We will get one solution, check how many boxes we needed, and add a constraint that we want one box less. We will do this until our formula is no longer satisfiable.

f = And(*constraints)

from pysat.solvers import Solver
with Solver(bootstrap_with=f) as solver:
  while solver.solve():
    model = solver.get_model()
    atoms = Formula.formulas((b for b in model if b > 0), atoms_only=True)
    boxes_needed = len(atoms)
    print("Solution with", boxes_needed, "boxes found")
    print(atoms)
    print("="*80)

    from pysat.card import CardEnc
    solver.append_formula(CardEnc.atmost(f.literals(boxes.values()), boxes_needed-1, vpool=Formula.export_vpool()))

It is important, when counting boxes, that we only extract the formulas/atoms that we created and not the ones created internally due to Tseyitin transformation. We count them and add a cardinality constraint, where we say, that we want at most one box less than we currently have used. An after only three iterations we get the following result:

Solution with 33 boxes found
[Atom(P(x=0, y=0, z=0)), Atom(P(x=0, y=1, z=0)), Atom(P(x=0, y=2, z=0)), Atom(P(x=0, y=1, z=1)), Atom(P(x=0, y=2, z=1)), Atom(P(x=0, y=1, z=2)), Atom(P(x=0, y=2, z=2)), Atom(P(x=1, y=0, z=0)), Atom(P(x=1, y=1, z=0)), Atom(P(x=1, y=2, z=0)), Atom(P(x=1, y=0, z=1)), Atom(P(x=1, y=0, z=2)), Atom(P(x=2, y=0, z=0)), Atom(P(x=2, y=1, z=0)), Atom(P(x=2, y=2, z=0)), Atom(P(x=2, y=0, z=1)), Atom(P(x=2, y=0, z=2)), Atom(P(x=3, y=0, z=0)), Atom(P(x=3, y=1, z=0)), Atom(P(x=3, y=2, z=0)), Atom(P(x=3, y=0, z=1)), Atom(P(x=3, y=0, z=2)), Atom(P(x=4, y=0, z=0)), Atom(P(x=4, y=1, z=0)), Atom(P(x=4, y=2, z=0)), Atom(P(x=4, y=0, z=1)), Atom(P(x=5, y=0, z=0)), Atom(P(x=5, y=1, z=0)), Atom(P(x=5, y=2, z=0)), Atom(P(x=5, y=0, z=1)), Atom(P(x=6, y=0, z=0)), Atom(P(x=6, y=1, z=0)), Atom(P(x=6, y=2, z=0))]
================================================================================
Solution with 32 boxes found
[Atom(P(x=0, y=0, z=0)), Atom(P(x=0, y=1, z=0)), Atom(P(x=0, y=2, z=0)), Atom(P(x=0, y=0, z=1)), Atom(P(x=0, y=0, z=2)), Atom(P(x=1, y=0, z=0)), Atom(P(x=1, y=1, z=0)), Atom(P(x=1, y=2, z=0)), Atom(P(x=1, y=1, z=1)), Atom(P(x=1, y=1, z=2)), Atom(P(x=2, y=0, z=0)), Atom(P(x=2, y=1, z=0)), Atom(P(x=2, y=2, z=0)), Atom(P(x=2, y=0, z=1)), Atom(P(x=2, y=2, z=1)), Atom(P(x=2, y=2, z=2)), Atom(P(x=3, y=0, z=0)), Atom(P(x=3, y=1, z=0)), Atom(P(x=3, y=2, z=0)), Atom(P(x=3, y=0, z=1)), Atom(P(x=3, y=0, z=2)), Atom(P(x=4, y=0, z=0)), Atom(P(x=4, y=1, z=0)), Atom(P(x=4, y=2, z=0)), Atom(P(x=4, y=2, z=1)), Atom(P(x=5, y=0, z=0)), Atom(P(x=5, y=1, z=0)), Atom(P(x=5, y=2, z=0)), Atom(P(x=5, y=2, z=1)), Atom(P(x=6, y=0, z=0)), Atom(P(x=6, y=1, z=0)), Atom(P(x=6, y=2, z=0))]
================================================================================
Solution with 31 boxes found
[Atom(P(x=0, y=0, z=0)), Atom(P(x=0, y=1, z=0)), Atom(P(x=0, y=2, z=0)), Atom(P(x=0, y=0, z=1)), Atom(P(x=0, y=0, z=2)), Atom(P(x=1, y=0, z=0)), Atom(P(x=1, y=1, z=0)), Atom(P(x=1, y=2, z=0)), Atom(P(x=1, y=2, z=1)), Atom(P(x=1, y=2, z=2)), Atom(P(x=2, y=0, z=0)), Atom(P(x=2, y=1, z=0)), Atom(P(x=2, y=2, z=0)), Atom(P(x=2, y=0, z=1)), Atom(P(x=2, y=0, z=2)), Atom(P(x=3, y=0, z=0)), Atom(P(x=3, y=1, z=0)), Atom(P(x=3, y=2, z=0)), Atom(P(x=3, y=1, z=1)), Atom(P(x=3, y=1, z=2)), Atom(P(x=4, y=0, z=0)), Atom(P(x=4, y=1, z=0)), Atom(P(x=4, y=2, z=0)), Atom(P(x=4, y=2, z=1)), Atom(P(x=5, y=0, z=0)), Atom(P(x=5, y=1, z=0)), Atom(P(x=5, y=2, z=0)), Atom(P(x=5, y=2, z=1)), Atom(P(x=6, y=0, z=0)), Atom(P(x=6, y=1, z=0)), Atom(P(x=6, y=2, z=0))]
================================================================================

Now we know that the lower bound on boxes is 31. But what does such a solution look like? Let us plot them using matplotlib, putting them all in a three-dimensional array

import matplotlib.pyplot as plt
import numpy as np

# Prepare some coordinates
x, y, z = np.indices((depth, depth, depth))

cubes = [((atom.object.x == x) & (atom.object.y == y) & (atom.object.z == z) )  for atom in atoms]

# Combine the objects into a single boolean array
voxelarray = cubes[0]
for cube in cubes[1:]:
  voxelarray |= cube

# Plot
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
ax.voxels(voxelarray, edgecolor='k')

ax.set(xticklabels=[], yticklabels=[], zticklabels=[])

plt.show()

And here is our trailer in glorius 3D:

Obviously the solution is not unique, as some stacks can be swapped and the whole configuration can be mirrored.

And just out of curiosity, if there were no gravity...

Solution with 26 boxes found
[Atom(P(x=0, y=0, z=0)), Atom(P(x=0, y=1, z=1)), Atom(P(x=0, y=2, z=1)), Atom(P(x=0, y=1, z=2)), Atom(P(x=0, y=2, z=2)), Atom(P(x=1, y=1, z=0)), Atom(P(x=1, y=2, z=0)), Atom(P(x=1, y=0, z=1)), Atom(P(x=1, y=0, z=2)), Atom(P(x=2, y=1, z=0)), Atom(P(x=2, y=2, z=0)), Atom(P(x=2, y=0, z=1)), Atom(P(x=2, y=0, z=2)), Atom(P(x=3, y=1, z=0)), Atom(P(x=3, y=2, z=0)), Atom(P(x=3, y=0, z=1)), Atom(P(x=3, y=0, z=2)), Atom(P(x=4, y=1, z=0)), Atom(P(x=4, y=2, z=0)), Atom(P(x=4, y=0, z=1)), Atom(P(x=5, y=1, z=0)), Atom(P(x=5, y=2, z=0)), Atom(P(x=5, y=0, z=1)), Atom(P(x=6, y=0, z=0)), Atom(P(x=6, y=1, z=0)), Atom(P(x=6, y=2, z=0))]
================================================================================
Solution with 25 boxes found
[Atom(P(x=0, y=2, z=0)), Atom(P(x=0, y=2, z=1)), Atom(P(x=0, y=0, z=2)), Atom(P(x=0, y=1, z=2)), Atom(P(x=1, y=2, z=0)), Atom(P(x=1, y=1, z=1)), Atom(P(x=1, y=0, z=2)), Atom(P(x=1, y=2, z=2)), Atom(P(x=2, y=2, z=0)), Atom(P(x=2, y=1, z=1)), Atom(P(x=2, y=0, z=2)), Atom(P(x=2, y=2, z=2)), Atom(P(x=3, y=2, z=0)), Atom(P(x=3, y=2, z=1)), Atom(P(x=3, y=0, z=2)), Atom(P(x=3, y=1, z=2)), Atom(P(x=4, y=1, z=0)), Atom(P(x=4, y=2, z=0)), Atom(P(x=4, y=0, z=1)), Atom(P(x=5, y=1, z=0)), Atom(P(x=5, y=2, z=0)), Atom(P(x=5, y=0, z=1)), Atom(P(x=6, y=0, z=0)), Atom(P(x=6, y=1, z=0)), Atom(P(x=6, y=2, z=0))]
================================================================================
Solution with 24 boxes found
[Atom(P(x=0, y=2, z=0)), Atom(P(x=0, y=2, z=1)), Atom(P(x=0, y=0, z=2)), Atom(P(x=0, y=1, z=2)), Atom(P(x=1, y=2, z=0)), Atom(P(x=1, y=1, z=1)), Atom(P(x=1, y=0, z=2)), Atom(P(x=1, y=2, z=2)), Atom(P(x=2, y=2, z=0)), Atom(P(x=2, y=1, z=1)), Atom(P(x=2, y=0, z=2)), Atom(P(x=2, y=2, z=2)), Atom(P(x=3, y=2, z=0)), Atom(P(x=3, y=1, z=1)), Atom(P(x=3, y=0, z=2)), Atom(P(x=4, y=1, z=0)), Atom(P(x=4, y=2, z=0)), Atom(P(x=4, y=0, z=1)), Atom(P(x=5, y=1, z=0)), Atom(P(x=5, y=2, z=0)), Atom(P(x=5, y=0, z=1)), Atom(P(x=6, y=0, z=0)), Atom(P(x=6, y=1, z=0)), Atom(P(x=6, y=2, z=0))]
================================================================================
Solution with 23 boxes found
[Atom(P(x=0, y=2, z=0)), Atom(P(x=0, y=2, z=1)), Atom(P(x=0, y=0, z=2)), Atom(P(x=0, y=1, z=2)), Atom(P(x=1, y=2, z=0)), Atom(P(x=1, y=1, z=1)), Atom(P(x=1, y=0, z=2)), Atom(P(x=1, y=2, z=2)), Atom(P(x=2, y=2, z=0)), Atom(P(x=2, y=1, z=1)), Atom(P(x=2, y=0, z=2)), Atom(P(x=3, y=2, z=0)), Atom(P(x=3, y=1, z=1)), Atom(P(x=3, y=0, z=2)), Atom(P(x=4, y=1, z=0)), Atom(P(x=4, y=2, z=0)), Atom(P(x=4, y=0, z=1)), Atom(P(x=5, y=1, z=0)), Atom(P(x=5, y=2, z=0)), Atom(P(x=5, y=0, z=1)), Atom(P(x=6, y=0, z=0)), Atom(P(x=6, y=1, z=0)), Atom(P(x=6, y=2, z=0))]
================================================================================
Solution with 22 boxes found
[Atom(P(x=0, y=2, z=0)), Atom(P(x=0, y=1, z=1)), Atom(P(x=0, y=0, z=2)), Atom(P(x=1, y=1, z=0)), Atom(P(x=1, y=0, z=1)), Atom(P(x=1, y=2, z=2)), Atom(P(x=2, y=0, z=0)), Atom(P(x=2, y=2, z=0)), Atom(P(x=2, y=0, z=1)), Atom(P(x=2, y=1, z=2)), Atom(P(x=3, y=0, z=0)), Atom(P(x=3, y=2, z=1)), Atom(P(x=3, y=1, z=2)), Atom(P(x=4, y=0, z=0)), Atom(P(x=4, y=2, z=0)), Atom(P(x=4, y=1, z=1)), Atom(P(x=5, y=1, z=0)), Atom(P(x=5, y=2, z=0)), Atom(P(x=5, y=0, z=1)), Atom(P(x=6, y=0, z=0)), Atom(P(x=6, y=1, z=0)), Atom(P(x=6, y=2, z=0))]
================================================================================
Solution with 21 boxes found
[Atom(P(x=0, y=1, z=0)), Atom(P(x=0, y=0, z=1)), Atom(P(x=0, y=2, z=2)), Atom(P(x=1, y=1, z=0)), Atom(P(x=1, y=2, z=1)), Atom(P(x=1, y=0, z=2)), Atom(P(x=2, y=0, z=0)), Atom(P(x=2, y=1, z=1)), Atom(P(x=2, y=2, z=2)), Atom(P(x=3, y=2, z=0)), Atom(P(x=3, y=0, z=1)), Atom(P(x=3, y=1, z=2)), Atom(P(x=4, y=0, z=0)), Atom(P(x=4, y=1, z=0)), Atom(P(x=4, y=2, z=1)), Atom(P(x=5, y=0, z=0)), Atom(P(x=5, y=2, z=0)), Atom(P(x=5, y=1, z=1)), Atom(P(x=6, y=0, z=0)), Atom(P(x=6, y=1, z=0)), Atom(P(x=6, y=2, z=0))]
================================================================================

NULLS NOT DISTINCT in SQLite Unique Indexes

2024-02-22T00:00:00+01:00

Recently I had the problem that I needed a unique index in an SQLite database, but some fields of my table had null values, which I needed to be considered equal, or NOT DISTINCT. Some database systems allow to specify the behaviour when creating indexes, but SQLite only supports the NULLS DISTINCT behaviour. In this article, we explore two ways to express the NULLS NOT DISTINCT behaviour in SQLite.

Not considering null values, two tuples are equal iff each of their components are equal. (1, 2, 3) is equal to (1, 2, 3). There are usually two ways to view null in databases: the absence of a value, or a value that exists but is not known.

In "absence of a value", we would consider (1, 2, null) to be equal to (1, 2, null), as we actually mean something akin to (1, 2), i.e. NULLS NOT DISTINCT. In "unknown value", we would consider (1, 2, null) to not be equal to (1, 2, null), as null represents an unknown but possibly or probably different value in either case, i.e. NULLS DISTINCT.

In our example we will consider an index on the table t with columns c1, c2, c3, where c1 is never null, but the other two columns can be.

Indexes on Expressions

One possible way (which I ultimately chose) is using indexes on expressions. Per column we choose a value which the column should never normally have. As the columns are untyped in SQlite, this is quite easy. You could choose a very large (or negative) number, or a random string. Again, some value you're fairly certain, your actual data is never going to be equal to. As an example, we choose -1, as my numbers are usually all positive. If you want to prevent accidentally using this value, you may exclude it using a CHECK-constraint.

Now instead of creating our index as

CREATE UNIQUE INDEX ON t (c1, c2, c3);

we create an index on coerced values:

CREATE UNIQUE INDEX ON t (c1, COALESCE(c2, -1), COALESCE(c3, -1));

The COALESCE function is variadic and returns its first non-null` argument. This means, our unique index does not compare the ``null values to each other (which are not equal), but our choses non-null representative (-1 in this case).

Problem 1: We do need a value per column that is not allowed. Often times it is very easy to find such a value. You could always use something that would be a type mismatch, or something that is malformed. But sometimes it might not be that easy and you might have to choose a different value for each column. Also, the value will be stored in the index and it should not be very large.

Problem 2: SQLite doesn't actually use this index for queries and operations other than insert. It will not make your SELECT statements faster. So you probably want to create another index, or multiple, depending on your access patterns, that are used for retrieval. Here you can just use the original index without the UNIQUE constraint:

CREATE INDEX ON t (c1, c2, c3);

Our uniqueness constraint is checked through the other index.

Partial Indexes

The second way is to use partial unique indexes for every null/not null combination. In this way we never allow null values to sneak into our index(es) and we are back to our base case.

CREATE UNIQUE INDEX ON t (c1) WHERE c2 IS NULL and c3 IS NULL;
CREATE UNIQUE INDEX ON t (c1, c2) WHERE c2 IS NOT NULL and c3 IS NULL;
CREATE UNIQUE INDEX ON t (c1, c3) WHERE c2 IS NULL and c3 IS NOT NULL;
CREATE UNIQUE INDEX ON t (c1, c2, c3) WHERE c2 IS NOT NULL and c3 IS NOT NULL;

These indexes will be used by your SELECT queries and we do not need a particular value per column to trick the index checker.

The problem with this method is obvious though, as we create $2^n$ indexes for $n$ nullable columns.

So what you really want depends on your use case. Using partial indexes is probably generally faster and less error-prone, and less memory-intensive. But you do need to manage multiple (exponentially many) indexes. If you're not worried about speed and your queries are already supported by the other indexes, and you only need a simple unique constraint over nullable columns, the indexed expressions might be the variant to use.

OAuth in GNOME Shell Extensions

2020-06-27T00:00:00+02:00

As you might know, I am the maintainer of the GNOME Twitchlive shell extensions. The Twitchlive panel allows you to see whether your favourite Twitch streamers are online or not.

A few months ago, Twitch started to require OAuth authentication for its endpoints and things started to go sideways. Now there are two ways to communicate with Twitch's API:

Authenticate your requests with a pre-shared application secret.
Obtain a client secret through user authentication and use this secret.

The first variant is only possible for server-to-server applications, as you're not allowed to distribute the application secret to users. So the second variant it is and we need to authenticate the user.

In the OAuth process you open a webpage (of the oauth provider) with a callback url. That webpage contains a login form and if you enter valid credentials, the webpage will redirect you to your given callback url, passing as additional information the authorization token you can use to make a valid API call.

To receive the token you need a webserver that you can redirect to. Even though GNOME's supposed to have a generic OAuth implementation it uses for its online services but it's not generic at all and you can't hook into that. You also can't create a webserver otherwise from within the GNOME Shell. Until a few weeks ago:

I've implemented a small mechanism by starting a python-based webserver through the extension that has just enough capabilities to receive the OAuth token and write it to a file. As far as I know, that's a world first. If you base OAuth of you extension on this work, give me a shout on twitter.

And now to the code which I think I boiled down right to the essentials (you'll also find it on github):

We need to open a browser with the callback within the extension:

function trigger_oauth() {
  const url = "https://id.twitch.tv/oauth2/authorize?response_type=token&client_id=" + client_id + "&redirect_uri=http://localhost:8877&scope=";
  GLib.spawn_command_line_async("xdg-open " + url);
  GLib.spawn_sync(null, ["python3", oauth_receiver,  oauth_token_path], null, GLib.SpawnFlags.SEARCH_PATH, null);
}

And the python3 reciver code:

from http.server import *;
from urllib.parse import *;
import sys;
import os.path;

page = """<html>
<head><title>Twitchlive GNOME Shell extension OAuth</title></head>
<body><script>var tokens=document.location.hash.substring(1);
document.write("<a href=\\"/tokens?" + tokens + "\\"> To finish OAuth-Process click here</a>");
</script></body>
"""

class handler(BaseHTTPRequestHandler):
  def log_requests(self):
      pass

  def do_GET(self):
      print(self.path)
      if self.path == '/':
          # initial call from twitch
          self.send_response(200)
          self.send_header("Content-Type", "text/html")
          self.end_headers()

          self.wfile.write(page.encode())
      elif self.path.startswith('/tokens'):
          # our own call
          code = parse_qs(urlparse(self.path).query)['access_token'][0]
          open(sys.argv[1], 'w').write(code)

          self.send_response(200)
          self.send_header("Content-Type", "text/plain")
          self.end_headers()

          self.wfile.write(b"Thank You. You can close this page.")
          sys.exit(0)

Note that the OAuth-Token is passed as the url fragment that doesn't show up in the actual query so you really need a browser to read that value.

And for now ... it works.

Podcast Time Machine

2020-01-23T00:00:00+01:00

Recently one of my favourite podcasts ended after seven years. I have only been listening to this podcast for two years though. There is five year backlog for me to listen to. But there are many other podcasts I want to listen to and I do like the anticipation of waiting for a new episode.

So I built the Podcast Time Machine (github) where you can enter a podcast url and a delay and you will get a new link where every podcast episode is delayed by that many days and all episodes that would lie in the future are filtered out. So stuff happpening a week in podcast time are happpening in a week of real time.

I am looking forward to my first episode of harmontown, when it comes out after a seven-year delay. Feel free to subscribe to your own delayed podcasts. I do not keep access logs so listen to your favorite stuff again (and again).

I bought a Cluster Hat

2020-01-19T00:00:00+01:00

I recently bought and installed a Cluster Hat.

Looks interesting

I don't know what I will do with it. All my plans I had before hinged on Haskell supporting ARMv6, which it doesn't. Let's see what happens in the future. I am currently looking into installing Docker on it and maybe host some service through my VPS from home.

Tuple Types, Notation, and Codings

2019-12-09T00:00:00+01:00

I wanted to write a bit about notation as I have seen students and programmers stumble quite a bit about this. What is "this"? When to write parentheses and commas in tuples and what does it mean.

Tuple Notation

Consider the set $S = \{a, b\}$. Now lets take the cross product of itself. $S \times S = \{(a,a), (a,b), (b,a), (b,b)\} = S^2$. By the same logic $S^3 = \{(a, a,a), (a, a,b), (a, b,a), (a,b,b), (b,a,a), (b,a,b), (b,b,a), (b,b,b)\}$. This is somewhat strange since $S^3 = S^2 \times S = S \times S^2$ meaning that these two should also be the same as $S^3$:

$\{(a, (a,a)), (a, (a,b)), (a, (b,a)), (a,(b,b)), (b,(a,a)), (b,(a,b)), (b,(b,a)), (b,(b,b))\} = \\ \{((a, a),a), ((a, a),b), ((a, b),a), ((a,b),b), ((b,a),a), ((b,a),b), ((b,b),a), ((b,b),b)\}$

Let's step back a bit and look at the identity, meaning $S^1$. We know that the power one is the identity but following the same logic as before we have the following: $S = \{a, b\} = S^1 = \{(a), (b)\}$. The parantheses do not matter and we are supposed to think of them as just as sequence of values. Let's take a look at the empty sequence: $S^0 = \{()\}$. Note that this is not the empty set but it is the neutral element for the cartesian product: $\{()\} \times \{a, b\} = \{((), a), ((), b)\} = \{a, b\}$.

What is going on here? The cartesian product is, as is the normal product, an extension of some kind of addition. In our case addition is concatenation of sequences and every value is a sequence of one element (singleton sequence). This is different to the singleton set which is distinct from the value it contains. $\{a, b\} \times \{c, d\} = \begin{Bmatrix}a \circ c \ & a \circ d\\b \circ c & b \circ d\end{Bmatrix}$ looks very much like the cross product for vectors and of course, the empty sequence is the neutral element (or identity) of that operation.

So actually we are supposed to take nested tuples as sequences of values that - in our mind - should be flattened. This also means, there is no one-tuple (it is the same as the value) and there ist just one zero-tuple.

By the way: this notation fits nicely as it just maps the cardinality of the sets. So if you just take $S=2$ you see that all the formulas just magically work out.

Why do we even use this tuple notation? This has to do with something we call coding.

Coding

I will define code the following way: A set is a code if no two different sequences of elements of that look the same when written down consecutively (this is a layman's definition that shall suffice here). Take the set $B = \{11, 100, 00, 001\}$. Both these sequences look the same when written consecutively: $(001,100)=(00,11,00)=001100$. Whether a set is a code is a special instance of the post correspondence problem. I leave it to the reader to define the mapping.

If the carrier set is a code, we do not need to use the tuple syntax. For the following reasons a set might trivially be a code: All symbols have the same length. There are no overlaps (no non-empty prefix of one symbol is a suffix of another) between the symbols.

So for the first set $S$ we do not need to use the tuple notation as it is always quite clear what is meant: $S^2 = \{aa,ab,ba,bb\}$. Using this notation we now need a new symbol for the empty sequence and we usually use $\varepsilon$ (epsilon). Note that concatenation follows these axioms: $a\circ\varepsilon = \varepsilon\circ a = a$ (neutral element) and $(ab)c=a(bc)$ (associativity). The concatenation operation forms a monoid under the carrier set (a semigroup with identity).

Tuple Types in Programming

Where does this confusion stem from? I think large part of it is that most programming languages do not properly support tuples in their type system in the way we want to use them in computer science theory. Programming langues do have tuple types but they are more rigid than what we want to use.

Let us look at python first:

# Python does have a native tuple type
>>> (3, 6)
(3, 6)
# We have an empty unit sequence
>>> ()
()
# Using the correct syntax we can nest this arbitrarily deep
>>> (((((),),),),)
(((((),),),),)
# We can construct one-tuples which are distinct from their value
>>> (3,) == 3
False
# The standard library has a sequence-product operation
>>> from itertools import product
# the cartesian product with itself works as we expect
>>> list(product([1,2,3], repeat=2))
[(1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (2, 3), (3, 1), (3, 2), (3, 3)]
# Having a zero-product returns the sequence that contains unit, which we expect
>>> list(product([1,2,3], repeat=0))
[()]
# Somewhat consistently, one-product returns the sequence of one-tuples but as seen above, this is not eqal to the original sequence
>>> list(product([1,2,3], repeat=1))
[(1,), (2,), (3,)]
# Also we do not have associativity
>>> list(product([1,2],product([3,4],[5,6])))
[(1, (3, 5)), (1, (3, 6)), (1, (4, 5)), (1, (4, 6)), (2, (3, 5)), (2, (3, 6)), (2, (4, 5)), (2, (4, 6))]
>>> list(product(product([1,2],[3,4]),[5,6]))
[((1, 3), 5), ((1, 3), 6), ((1, 4), 5), ((1, 4), 6), ((2, 3), 5), ((2, 3), 6), ((2, 4), 5), ((2, 4), 6)]

I also want to take a look at Haskell which is also interesting.

-- Haskell has native tuple types
> :t (3,5)
(3,5) :: (Num a, Num b) => (a, b)
-- But there is no singleton tuple type
> :t (5)
(5) :: Num p => p
> (5) == 5
True
-- We have the unit type that is inhabited with the single instance unit which we also can not nest
> :t ()
() :: ()
> :t ((((()))))
((((())))) :: ()
-- Checking for associativity is a type error as both nestings are distinct types and can not be compared via equals
Prelude> (1,(2,3)) == ((1,2),3)
<interactive>:6:2: error:
Prelude> :t (1,(2,3))
(1,(2,3)) :: (Num a1, Num a2, Num b) => (a1, (a2, b))
Prelude> :t ((1,2),3)
((1,2),3) :: (Num a, Num b1, Num b2) => ((a, b1), b2)

The type system does not allow us to even construct a type-safe product function as it is in python as the tuple size of the result sequence is dependent on the value of the argument (something like a -> Int:Val -> (a,)*Val). This would need dependent types. There is a generic function for the cartesian product with the type [a] -> [b] -> [(a, b)] which, as we have seen from the interactive session, would not obey the associativity law either.

Conclusion and Context

We have seen that the tuple notation is just that: notation. It is not used for structure, only for disambiguity. If you want structure, you need to use uninterpreted functions $t(1,t(2,3)) \neq t(t(1,2),3)$ that are not flattened. But programming languages and the type systems do see a structural difference in the tuple types. The 3-product in Haskell could have, depending on the implementation, three different types: [a] -> [b] -> [c] -> [(a,(b,c))] or [a] -> [b] -> [c] -> [((a,b),c)] or [a] -> [b] -> [c] -> [(a,b,c)] where we would expect all types to be the same.

Why did this article come to pass? Two reasons. During the database lectures we wanted to write up some database state (a set of rows for each table, a row being a sequence of column values) and while one table had three columns ($\{(1,2,3),(4,5,6)\}$) one table only had one column and for symmetry $\{(1),(2),(3)\}$ was brought to paper which I remarked to be at least confusing for students as it might convey something not meant (there being a difference between the singleton tuple and the value).

The second example comes from computer science didactics, a module for the aspiring teachers. In a homework where some lesson had to be designed, one student had the topic of languages and grammars. In an example of a regular language they took this as an example alphabet: $A=\{la,li,lu\}$. This is fine as the symbol of the alphabet are just syntax and are only of interest (as I've written above), in terms of whether the alphabet is a code or not and therefore which notation is used for concatenation operations. Now on a worksheet the following question was asked: $lul \in A^*$. And this, I would claim, is a type error. The kleene closure of the alphabet only contains sequences of symbols from the alphabet (including the empty sequence) but "lul" is no sequence of symbols from the alphabet so it can not be of the same type (sequence from the alphabet) as the set (set of sequences from the alphabet). As the original alphabet was a code, all sequence can be written consecutively but it is unclear how "lul" can be deconstructed into elements of the alphabet. A possibiliy is, that the original alphabet was $\{l,a,i,u\}$ and $A$ was just a language over that alphabet but that was not written. For teachers, I believe, formalisms in that area are quite important.

Java Primitive Types are bad for Learners

2018-11-07T00:00:00+01:00

This year I am teaching Java to first semester students. My own Java introduction has happened seven years ago. As I am quite interested in principles of programming languages I try to convey my enthusiasm for the topic and base my didactic methods around that. It's cool what we do. Let's talk about it.

I see the problems in a lecture for first semester students as they have yet to learn any theoretical basis for object oriented programming, that would be useful. In my oppinion one should start with the theoretical basis, like class hierarchies and the object model. But students only learn about partial orders (like a class hierarchy) later in the first year. So ideally (in my mind at least), a computer science education is without a computer for the first semester.

And because there is no theoretical basis, the approach taken by the professor is quite practical, and just barely supported by theory (wherever possible, but that's not much). This means that after the first lesson, students already know what a Hello World program looks like and they can fiddle with it. Even though they do not know what a class is, a static method, even methods or functions at all.

This leads to further problems down the road. We have an online platform where students have to solve programming assignments during seminars and as homework. But since they don't know about classes and class methods (or types or return statements for that matter), we can't have them implement classes, methods, or interfaces as homework and have to have them write output into the main method. That's a bad start, when you consider programming best practices.

Ideally one would like to start implementing small methods without function calls that implement some interface and then go on in creating more complex programs later on. But the problem is, that there is no way to run the java code without a main method. And even if you provide one, then the students can not really practice.

This shows how necessary a REPL or interactive interpreter is.

But Java has a few more problems that are quite bad for learners and can lead to confusion and misunderstanding.

Some Types are Keywords

For legacy reasons, Java has two kinds of types. There are primitive types that correspond in some manner to machine types, and the reference types that form the class hierarchy. Examples of primitive types would be int or float or void (Unit - only allowed as method return type). In general it's also the case that operators work on primite types but do not for reference types (exceptions apply). Those primitive types are builtin insofar that their type names are keywords as well.

As a homework for the second week I gave the following task: write a program that reads a number between 0 and 100 from the standard input (they don't know about standard input yet, and I'd prefer passing arguments as command line parameters but was overruled) and write a 10 character progress bar, visualizing the progress that correspond to the input number. This is my solution I worked from:

import java.util.Scanner;

class Progress {
  public static void main(String[] args) {
    Scanner input = new Scanner(System.in);

    int progress = input.nextInt() / 10;
    int rest = 10 - progress;

    while (progress > 0) {
      System.out.print("|");
      progress = progress - 1;
    }
    while (rest > 0) {
      System.out.print("-");
      rest = rest - 1;
    }
  }
}

Since this is quite complicated for beginners, I've given them the basic structure and removed all keywords, replacing them with % and replacing all variable names with #. Resulting in this:

import java.util.Scanner;

% Progress {
  % % % main(String[] #) {
    Scanner # = % Scanner(System.in);

    % # = #.nextInt() / 10;
    % # = 10 - #;

    % (# > 0) {
      System.out.print("|");
      # = # - 1;
    }
    % (# > 0) {
      System.out.print("-");
      # = # - 1;
    }
  }
}

And as you can see, not entirely all type information is removed by replacing the keywords with #. If all types were reference types, we'd have a program that looked more like this:

% Progress {
  % % Unit main(String[] #) {
    Scanner # = % Scanner(System.in);

    Integer # = #.nextInt() / 10;
    Integer # = 10 - #;

    % (# > 0) {
      System.out.print("|");
      # = # - 1;
    }
    % (# > 0) {
      System.out.print("-");
      # = # - 1;
    }
  }
}

Now doesn't this look quite a bit better? Some types are introduced as keywords but all user-created ones are not (and by user-created, I also mean the standard library). This is an inconsistency that is quite hard to explain away, even more so if you have yet to introduce the students to the vocabulary to explain this.

There are even more problems quite soon. Once conditionals are introduced, we have the next problem. The language construct if () then {} else {} only works with the bool type as conditionals. So primitive types are necessary to use the language constructs for conditional execution (while as well, but the numeric comparison follows somewhat naturally).

I do not know whether we will get this far, but for type parameters (for for polymorphic types, like containers) the primitive types don't work either.

All in all, I'd say Java would need to do away with primitive types and consistently use reference types. But for this to work properly with the operators, one would probably also need to add operator overloading. Once you're there, you could think long and hard about whether you'd really need the new keyword for instance construction.

And of course, one would want to introducing a proper Unit type. But since in Java all types are inhabited by the value null (who the hell had that idea?), this would be useless. So I will allow keeping the void keyword.

Finding Trade and Exchange Possibilities with Prolog

2018-09-07T00:00:00+02:00

A friend recently had an interesting problem for me: There's a game for mobile phones where you have to exchange or transmute wares into each other, in order to create some strange supply chain.

Example of an earlier level.

A very complex later example.

The game is bleentoro, image copyright by them.

In this example (an earlier level) the goal is to create fish. We can transmute or exchange (I will only write about exchange from now on) an apple and a pair of cherries for a fish. Apple and cherry we have, or are created from thin air, in the context of the game. As you can see in the other example, this can get very complex.

The question is, given some input we have, in what order do we have to use the stations for exchange to create some output. How do we best model this problem?

Modeling the Problem

From a previous article we already know that we can encode numbers in Prolog terms using Peano numbers. Using these we can match terms that represent at least or exactly some number. Exactly 3 would be s(s(s(z))) while at least 3 would be s(s(s(X))) with the variable X representing the rest of the term for some $n$ so that $3 + x = n$.

Let's start with a small example: we have an aunt that gives us apples. We have bob who is willing to exchange an apple for two berries. And we have a cousin who will bake a cake if we give them two apples and five berries.

Let's formalize our rules in some way:

a: / -> 1A
b: A -> 2B
c: 2A, 5B -> 1C

We model this as some kind of inventory(A, B, C) with some slots for Peano encoded numbers. Meaning that inventory(s(z), z, s(C)) means an inventory of exactly one apple, zero berries, and at least one cake. Lucky us.

Let's define our possible exchanges as Prolog facts with exchange(Kind, Pre, Post) meaning we use the exchange method Kind and we get from inventory state Pre to inventory state Post. If we lose an item, we have it in Pre but don't in Post. If we gain one, it's the other way around. Of course, the rest of our inventory should stay the same.

exchange(a, inventory(A, B, C), inventory(s(A), B, C)).
exchange(b, inventory(s(A),B,C), inventory(A, s(s(B)), C)).
exchange(c, inventory(s(s(A)), s(s(s(s(s(B))))), C), inventory(A, B, s(C))).

As we can see, we can encode basically any numeric condition that uses only "just exactly" and "at least", as well as addition and subtraction and "setting" fields to specific values. If we have to give at least three apples, but all of them, to get a cake, the rule would be exchange(c, inventory(s(s(s(_))), B, C), inventory(z, B, s(C)))..

Programming the Solver

We want to create a predicate trades(Start, Goal, Path) where we give a Start inventory and a Goal inventory and want to get a Path of exchanges. The most obvious approach is the following:

trades(Goal, Goal, []).
trades(Start, Goal, [H|T]) :- exchange(H, Start, Temp), trades(Temp, Goal, T).

So we're done when our Goal is reached. And if not, we have to try some exchange. We try it and we quickly learn that...

?- trades(inventory(z, z, z), inventory(_, _, s(z)), Path).
ERROR: Out of local stack

... it doesn't work.

The problem is, that calling the exchange predicate in the recursive call always tries the first rule first and since that basically always works, we get into an infinite loop. Let's reorder the rules and put the generating rule first.

?- trades(inventory(z, z, z), inventory(_, _, s(z)), Path).
ERROR: Out of local stack

It still doesn't work. Let's look at the trace.

[trace]  ?- trades(inventory(z, z, z), inventory(_, _, s(z)), Path).
 Call: (8) trades(inventory(z, z, z), inventory(_700, _702, s(z)), _712) ? creep
 Call: (9) exchange(_960, inventory(z, z, z), _984) ? creep
 Exit: (9) exchange(a, inventory(z, z, z), inventory(s(z), z, z)) ? creep
 Call: (9) trades(inventory(s(z), z, z), inventory(_700, _702, s(z)), _962) ? creep
 Call: (10) exchange(_978, inventory(s(z), z, z), _1002) ? creep
 Exit: (10) exchange(b, inventory(s(z), z, z), inventory(z, s(s(z)), z)) ? creep
 Call: (10) trades(inventory(z, s(s(z)), z), inventory(_700, _702, s(z)), _980) ? creep
 Call: (11) exchange(_1000, inventory(z, s(s(z)), z), _1024) ? creep
 Exit: (11) exchange(a, inventory(z, s(s(z)), z), inventory(s(z), s(s(z)), z)) ? creep
 Call: (11) trades(inventory(s(z), s(s(z)), z), inventory(_700, _702, s(z)), _1002) ? creep
 Call: (12) exchange(_1018, inventory(s(z), s(s(z)), z), _1042) ? creep
 Exit: (12) exchange(b, inventory(s(z), s(s(z)), z), inventory(z, s(s(s(s(z)))), z)) ? creep
 Call: (12) trades(inventory(z, s(s(s(s(z)))), z), inventory(_700, _702, s(z)), _1020) ? creep
 Call: (13) exchange(_1040, inventory(z, s(s(s(s(z)))), z), _1064) ? creep
 Exit: (13) exchange(a, inventory(z, s(s(s(s(z)))), z), inventory(s(z), s(s(s(s(z)))), z)) ? creep
 Call: (13) trades(inventory(s(z), s(s(s(s(z)))), z), inventory(_700, _702, s(z)), _1042) ? creep
 Call: (14) exchange(_1058, inventory(s(z), s(s(s(s(z)))), z), _1082) ? creep
 Exit: (14) exchange(b, inventory(s(z), s(s(s(s(z)))), z), inventory(z, s(s(s(s(s(s(z)))))), z)) ? creep
 Call: (14) trades(inventory(z, s(s(s(s(s(s(z)))))), z), inventory(_700, _702, s(z)), _1060) ? creep
 Call: (15) exchange(_1080, inventory(z, s(s(s(s(s(s(z)))))), z), _1104) ? creep
 Exit: (15) exchange(a, inventory(z, s(s(s(s(s(s(z)))))), z), inventory(s(z), s(s(s(s(s(s(z)))))), z)) ? creep
 Call: (15) trades(inventory(s(z), s(s(s(s(s(s(z)))))), z), inventory(_700, _702, s(z)), _1082) ? creep
 Call: (16) exchange(_1098, inventory(s(z), s(s(s(s(s(s(z)))))), z), _1122) ? creep

We're still running into an infinite loop since we're just getting apples and trading them for berries.

There's a trick if our solution is a list. If the length(List, Length) predicate is called with both predicates free (length(L, _) for example), we get lists of increasing length as solutions. So if we can bind our solution to lists of increasing length, we won't get infinite loops of apple-berry-exchanges without ever trying short solutions.

So our query now looks like this:

?- length(Path,_), trades(inventory(z, z, z), inventory(_, _, s(z)), Path).
Path = [a, b, a, b, a, b, a, a, c] ;
Path = [a, b, a, b, a, a, b, a, c] ;
Path = [a, b, a, b, a, a, a, b, c] ;
Path = [a, b, a, a, b, b, a, a, c] ;
Path = [a, b, a, a, b, a, b, a, c] ;
Path = [a, b, a, a, b, a, a, b, c] ;
Path = [a, b, a, a, a, b, b, a, c] ;
Path = [a, b, a, a, a, b, a, b, c] ;
Path = [a, b, a, a, a, a, b, b, c] ...

As it turns out, there are a few ways to do it, but nothing shorter than 9 trades. We can even look, whether we have anything left over after we got our cake:

?- length(Path,_), trades(inventory(z, z, z), inventory(A, B, s(z)), Path).
Path = [a, b, a, b, a, b, a, a, c],
A = z,
B = s(z) .

Yeah. As it turns out, we have a berry to spare.

Conclusion

This is how you do it. And of course, you could simulate a shop with that, as one inventory slot could be the number of coins. If the inventory gets too large, you can split it into item types so you don't have to copy every unchanged item from the pre-condition to the post-condition. Then you can copy a term that might represent any number of items.

But beware: this algorithm (with Prolog's evaluation scheme) has not so good complexity. For a solution of length $n$ and $m$ rules it has a run time of $O(m^n)$. We create every possible solution list permutation but break early, if no rules are applicable.

Depending on our instantiation of Start and Goal, this algorithm doesn't terminate (if the system isn't terminating, the algorithm isn't either). Our system doesn't terminate as we can always apply a again and again. We're creating increasingly longer partial solutions which might turn out to be no solution at all. If there is no solution, you won't know. The algorithm just tries longer and longer lists trying to find a solution.

If you have any other conditions on your solution (some exchanges might be limited or necessary) you can put most of them after the trades call. Some conditions, like membership, can be put before.

Let's say we have a terminating system by taking away the a rule. If our Start is fully instantiated, then our algorithm terminates. Let's try another query and find out, with how many apples we need to start to get a cake:

?- trades(inventory(A, z, z), inventory(_, _, s(z)), Path).
ERROR: Out of local stack

Through the unification rules, it is possible that a rule creates terms on the left side of the rule. In this case, the algorithm tries to give us increasing amounts of berries to match that with the starting condition (only rule b is applied in reverse, always leaving us with one cake, which doesn't match the starting condition). There we need the length predicate.

?- length(Path,_), trades(inventory(A, z, z), inventory(_, _, s(z)), Path).
Path = [b, b, b, c],
A = s(s(s(s(s(_2334))))) .

We need at least five apples for a cake. But we knew that.

To recap:

We saw again how useful it is, to encode the natural numbers in a term structure.
A solver is easily written but the algorithm isn't fast.
The rules can be applied in both directions because of Prolog's evaluation scheme. This leaves queries that would intuitively work divergent.
By using the length predicate we can simulate breadth-first evaluation, while the default evaluation scheme is depth-first.

I think, there's a different approach using linear programming. I might try that in the future.

Haskell Countdown Solver

2018-08-27T00:00:00+02:00

I love watching 8 Out Of 10 Cats Does Countdown. It's a british panel show where a few comedians solve word puzzles and mathematics puzzles. It's based on the game show Countdown that's been running since 1982.

And I also like the host Jimmy Carr very much.

The Puzzles

The show consists of three different puzzles:

In the Letters round a team (or single contestant) picks 9 consonants and vowels from their respective piles. After the team chooses from one of both piles, the letter is revealed and put up on the board. They then choose again until 9 letters are on the board. After all letters are on the board, the teams have 30 seconds to form a word out of as many of the letters as possible. The team with the longest word scores that many points.

Letters Round

In the Conundrum a list of letters that already spell out a funny group of words is given by the host and the contestants try to find a 9 letter anagram. The first team to buzzer in the correct answer gains 10 points. This is the final round of the game.

Conundrum

In the Numbers round a team (or single contestant) picks 6 numbers from two piles. A random number is picked and the contestants have to create a mathematical formula out of the picked numbers that equal the target number. The specific rules follow.

Numbers Round

According to Wikipedia there are 6 columns of 4 numbers (4 rows). One column contains the numbers 25, 50, 75, and 100 and the other 5 columns contain the numbers 1 to 10 twice each. That's not exactly the same as on the show, as there appear to be 7 columns of 4 numbers but not every stack has all 4 number cards.

Contestants choose a total of 6 numbers from the "small" and "large" columns. The usual pick would be 2 to 3 "large" numbers and the rest small ones.

Then a random number between 100 und 999 is generated. The contestants try to create a mathematical formula that has the random number as the result. In the formulare only the following operations are allowed:

Addition
Multiplication
Subtraction (that does not result in a negative number)
Division (that results in an integer)

The contestants do not have to use all the numbers to get the target number.

Finished Numbers Round

Since I am really not good at this, I wrote a solver for the numbers game in Haskell.

Solver

How do we go about this problem? First of all, we have to decide on a data structure. Let's start with a traditional term structure where a Term is a binary operation and two terms, or an integer value. An Operation is the function mapping two integers to a third, and a string to pretty-print the operation. The mapping is not total and might fail.

data Term = Op Operation Term Term | Val Integer
data Operation = Operation {fun :: (Integer -> Integer -> Maybe Integer), name :: String}

instance Show Term where
  show (Val x) = show x
  show (Op op l r) = "(" ++ (show l) ++ (name op) ++ show r ++ ")"

Now we have to define our operations and what they should do. Since we're using Maybe, and it's a monad, we can use monadic notation where it suits us:

operations = [  Operation (\x y -> return (x + y)) "+"
             ,  Operation (\x y -> return (x * y)) "*"
             ,  Operation (\x y -> guard (x > y) >> return (x - y)) "-"
             ,  Operation (\x y -> guard (x `mod` y == 0) >> return (x `div` y)) "/"
             ]
eval :: Term -> Maybe Integer
eval (Val v) = Just v
eval (Op o l r) = do l' <- eval l
                     r' <- eval r
                     (fun o) l' r'

As we can see, the preconditions to the subtraction and division are encoded using the guard function. If the function returns true, the whole operation returns Nothing, otherwise it returns Just the value.

Since Haskell uses lazy evaluation, we can define a list of all possible terms given a list of integers and filter for the ones that evaluate to the correct number.

As we create a Term we have to be careful to not use any of the numbers again. So we will split our numbers in two sets. One that can be used in the left Term of an Operation and the rest can be used in the right Term. We will create all terms using our definition. A Term is either the Value of one of the numbers for that subterm, or an operation with some numbers reserved for the left term and some numbers reserved for the right term.

terms :: [Integer] -> [Term]
terms nums =  do  n <- nums -- choose a number
                  [Val n] -- one solution
              ++
              do  op <- operations -- choose an operation
                  (l, r) <- subset_split nums -- choose a split
                  guard $ l /= []
                  guard $ r /= []
                  ls <- (terms l) -- choose a term for the left side
                  guard (eval ls /= Nothing)
                  rs <- (terms r) -- choose a term for the right side
                  guard (eval rs /= Nothing)
                  [Op op ls rs] -- one solution

For performance reasons we already evaluate the generated subterms in order to check whether the operations are valid. There are a lot of terms that can not be evaluated and we want to throw them out as early as possible (or not even generate them).

We haven't defined subset_split yet. The idea is, to just split the list of numbers in two groups in every possible way:

subsets :: [Integer] -> [[Integer]]
subsets []  = [[]]
subsets (x:xs) = subsets xs ++ map (x:) (subsets xs)

subset_split nums = do l <- subsets nums
                       return (l, nums \\ l)

Now all that remains is to generate all terms given a set of numbers, evaluate them, filter out the ones that don't have the correct value, and return the first one:

solve :: [Integer] -> Integer -> Maybe Term
solve nums target = let possible_solutions = map (\x -> (x, eval x)) (terms nums)
                        solutions = filter (\x -> (snd x) == (Just target)) possible_solutions
                    in listToMaybe $ map fst solutions

And in our main we only read the list of numbers and the target as arguments and print the solution:

main = do args <- getArgs
          let nums = (read (args !! 0))::[Integer]
              target = (read (args !! 1))::Integer
          print $ solve nums target

And there we go:

$ ./countdown "[100, 75, 2, 10, 3, 8]" 746
Just (8+(3+((75*(100-2))/10)))

Our solution is $8+3+{{75*(100-2)} \over 10}$ while the solution from the show was $10*75 - {8 \over 2}$.

As you can see, we do not always get the easiest solution but we usually have a solution quite fast. This one took about a tenth of a second. The 30 seconds given to the contestant are usually quite enough. To make sure we had the "nicest" solution, we probably would need to generate all terms and sort them by size. The smallest one should be the nicest one.

Lost in Localisation - Ordering a Taxi while on Holiday

2018-08-11T00:00:00+02:00

I was on holiday in England this summer. I arrived in Worthing by bus from London, had a full day there, and wanted to take the train the following day to Bristol. The bus route from my BnB to the station was inconvenient and expensive. Ordering a taxi was my next idea. I wanted to do that without human interaction the evening before. This is the story of how that did not work.

I name the taxi company in this article. This is not to shame them. I hope they just fix the issues, which I believe present in many applications, because localisation is hard.

Quite a Promise

The Mobile App

Before analyzing why ordering a taxicab through their website did not work, I wanted to try the mobile app. I was hopeful, considering their promise that I could "book a car in seconds and don't even need to ring us!".

As it turns out, this app is not available through the German Google Play Store. This means, you won't find it when you search for it and if you have the correct URL, you can't install the app through the web interface on a device that is connected to the German Google Play Store (so all of mine).

But this is not a problem since you can download any APK that is available from any app store through a third-party service like APKPure (unendorsed). So knowing the name of the app in the Play Store I did that and installed the app.

That is the App

You're presented with a registration screen where you have to enter your phone number to receive a confirmation SMS. My (international) phone number didn't take and I was not able to complete the process. We will later see why. Being unable to register via the app, I return to the web site.

The website looks quite modern

The Web Site

The first part of the order form

As you can see, to order a taxi you first have to enter where you want to be picked up and where you want to go. The system then calculates the distance between both points, estimates the cost, and then you can go on to the next step.

In order to calculate the distance, the web site uses the Google Maps API to create a route between both points and calculates a cost. Here's the corresponding code from the site:

journey['distance'] = response.routes[0].legs[0].distance;
journey['duration'] = response.routes[0].legs[0].duration;
journey['miles'] = ( journey['distance'].value * 0.000621371192 );
journey['price'] = CurrencyFormatted(2.90);
journey['total'] = CurrencyFormatted( journey['price']
                              + 1.90 * ( journey['miles'] - 0.2 ));

journey['start_address'] = response.routes[0].legs[0].start_address;
journey['end_address'] = response.routes[0].legs[0].end_address;

This looks okay. It is to note that the site takes the addresses returned by google as the canonical start and end of the journey. So in principle, you could enter the name of a shop or restaurant as the target location of your taxicab ride and as long as google knows it, the taxi service is booked to the actual and correct address.

Then there's another step of validation: Since the taxicab company only operates in Worthing in the south of England, they do not want to accept rides that are outside a specific area. They want to achieve this by parsing the post code from the canonical google response and compare it to a set of given post codes that correspond to the Worthing area: journey['start_postcode'] = journey['start_address'].match(/([A-Z]?\d(:? \d[A-Z]{2})?|[A-Z]\d{2}(:? \d[A-Z]{2})?|[A-Z]{2}\d(:? \d[A-Z]{2})?|[A-Z]{2}\d{2}(:? \d[A-Z]{2})?|[A-Z]\d[A-Z](:? \d[A-Z]{2})?|[A-Z]{2}\d[A-Z](:? \d[A-Z]{2})?),\s*UK$/)[0]. That's quite a large regular expression and I give you a visualisation powered by Regexper:

Visualisation of the regular expression

This does correspond in some fashion (but not exactly [1]) to the valid UK post codes followed by ", UK" and the end of the input.

I give you part of the google response for a ride between some random house and the train station and let you figure out why this step fails for me:

{ "routes" : [
      { "legs" : [
            {
               "distance" : {
                  "text" : "1,7 km",
                  "value" : 1654
               },
               "duration" : {
                  "text" : "5 Minuten",
                  "value" : 292
               },
               "end_address" : "Worthing BN11 1AA, Vereinigtes Königreich",
               "start_address" : "Harrow Rd, Worthing BN11 4RB, Vereinigtes Königreich"
            }
      ]}
]}

Google does return the post code correctly, but since my browser is set to German, Google's response is in German as well. The regular expression only matches strings ending in "UK", while my German response ends in "Vereinigtes Königreich" ("United Kingdom" in German).

The fix seems easy enough. Just set the browser language to UK English and do the whole thing again and there we go:

Success?

Okay, good. Let's book the taxi.

Now we just have to enter our name, email address, and phone number and we're good and finally can go by taxi wherever we want to go (within the greater Worthing area). As I enter my phone number, starting with +49 or 0049 for Germany, followed by three digits for the service provider [2], and seven or eight digits for the user, it dawns on me that it might not be that easy (as if it had been up to this point) and I take another glance at the code:

function validateTelephone(telephone){
  var re = /^(?:\W*\d){11}\W*$/;
  return re.test(telephone);
}

Oh oh. Okay. Yeah. About that...

My phone number does not fit that scheme (while 00000000000 does). If I get any confirmation code I need to respond to, or they want to call back for some additional questions, they won't be able to reach me. This is probably also the reason why the app did not work and why I couldn't receive an SMS with a confirmation code.

I won't be able to order a taxi online. To recap the two main reasons why:

Because my browser is set to something else than English, the website can not verify that the cab journey takes place within the serviced area. Solution: Google provides GPS coordinates for the legs of the journey. One can easily check whether they are contained within a rectangle or convex polygon.
Because I have an international phone number, I can't interact with the parts of the process that depend on my cellphone. Solution: Allow for international phone numbers or make the process not depend on them.

In the end I called them by phone, which took a whopping 25 seconds and the taxi arrived on time to bring me to the train station.

[1]	According to Wikipedia, the UK government has provided a regular expression that can be used for the purpose of validation: `^([Gg][Ii][Rr] 0[Aa]{2})\|((([A-Za-z][0-9]{1,2})\|(([A-Za-z][A-Ha-hJ-Yj-y][0-9]{1,2})\|(([A-Za-z][0-9][A-Za-z])\|([A-Za-z][A-Ha-hJ-Yj-y][0-9]?[A-Za-z])))) [0-9][A-Za-z]{2})$`

[2]	It's a little bit more complicated but sufficient for this story.

Prolog Programs, Order of Evaluation, and Functional Equivalents

2018-07-18T00:00:00+02:00

While teaching Prolog this semester I had a few thoughts about Prolog programs and their equivalent programs in a functional programming language like Haskell.

The Peano Naturals

The Peano numbers are a unary number system for positive natural numbers where the zero is represented by a Z (or 0, or any atom - it doesn't matter) and every successor of a natural number N is the function S applied to N. We would therefore represent the natural number 3 with the term S(S(S(Z))). To look at it differently: in the Peano axioms, the naturals are defined as the set that contains zero and a closure over the successor function. In haskell we would define the type of the naturals like this:

data N = Z | S N

In Prolog we don't need a type definition for the naturals, as terms are generated without types. Now we want a predicate that checks whether our input is a natural number as we've defined it. The idea is that the term is a natural, iff either the term is a Z or it has an S as its root and its parameter is a natural. In Prolog we would write it intuitively like this:

naturals(z).
naturals(s(X)) :- naturals(X).

And indeed this checks whether our input is a natural number. The query naturals(s(s(z))). returns true while naturals(foo). does not. Writing down this program in Haskell is not very useful since the type system does not allow us to have anything else but a natural as the function parameter but let's pretend:

naturals v = case v of
              S x -> naturals x
              Z -> True
              _ -> False

And of course, calling naturals $ S $ S Z in haskell has the correct type and returns true. But this haskell program only corresponds to the case where we actually call the Prolog predicate with a bound parameter. We could call the prolog predicate with a free binding pattern like naturals(X). and get bindings for X for every solution, so every natural number. The corresponding Haskell programm would be:

naturals_f  = [ Z ] ++ map S naturals_f

This might need some explaining: The list corresponds to some notion of non-determinism. In Prolog the non-determinism is baked into the language due to backtracking from failed predicate evaluations and us asking for more answers. The Prolog program essentially says: Z is a solution. Everything with an S is a solution, if the "content" of the S is a solution as well. That's the same for our haskell programm. In our list of solutions we know that Z is in it and for every solution we have, adding an S leads to another one. And evaluating naturals_f starts to calculate the infinite list of natural numbers.

Now consider the following Prolog program where we switched the order of the clauses:

naturals(s(X)) :- naturals(X).
naturals(z).

This still works nicely for the bound case where we call naturals(s(s(z)). But if we now call naturals(X) we quickly get ERROR: Out of local stack. The natural intuition might be that the order of clauses does not matter but it does in this case. Writing down the equivalent Haskell programm we are able to see why:

naturals_f  = map S naturals_f ++ [ Z ]

We have the recursion before we have any solution. Well, the solution is easy: just write the simple case in the first line and the recursive case in the second. But let's consider another problem.

Repeated Printing

Consider the problem of repeatedly printing a character:

repeat(0, _).
repeat(N, C) :- write(C), NN is N - 1, repeat(NN, C).

This looks very correct and is very wrong.

?- repeat(5, f).
fffff
true ;
ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff...

Now we see why it's nice to represent the naturals als Peano numbers. After hitting 0 there's a choice point left and asking for another solution leads to the negatives and printing our character in an infinite loop. So let's augment the second rule to prevent us from going into the negatives.

repeat(0, _).
repeat(N, C) :- N > 0, write(C), NN is N - 1, repeat(NN, C).
% query:
?- repeat(5, f).
fffff
true ;
false.

This looks better but we still have a choice point left over which might be not so nice. The problem is that the matched patterns are not disjoint and after the end of the recursion there's still a pattern left over that fits. We can't encode that N should be larger than 0 in the head of the rule. Remember that we can match naturals larger than 0 (or any other number) in the rule head using Peano numbers - that's why they are so cool.

But how do we fix our problem with the left-over choice point? There are two ways we can do it. We can either introduce a cut or reorder the rules.

% Variant 1a: The order doesn't matter but we introduce a cut
repeat(0, _) :- !.
repeat(N, C) :- N > 0, write(C), NN is N - 1, repeat(NN, C).

% Variant 1b: If we promise to be good and only call repeat with a positive numeric argument, we can even leave out the range check
repeat(0, _) :- !.
repeat(N, C) :- write(C), NN is N - 1, repeat(NN, C).

% Variant 2: The order does matter
repeat(N, C) :- N > 0, write(C), NN is N - 1, repeat(NN, C).
repeat(0, _).

I personally think that the variant without the cut is nicer. So for terminating rules, it might be better to have the "smaller" rule last as to not leave choice-points hanging around. Of course, this only works if the matched terms in the rules are proper subsets of each other so that we actually can order the rules in that fashion. In general, that might not be possible.

Conclusion

The order of rules can (in theory) not affect termination. But if our rules are already non-terminating for our query, the order and timeliness of results does matter. Then order does matter.
Disjoint rule heads are important to not have left-over choice points.
If rules are terminating but the rule heads are not disjoint, they still should be ordered carefully.
Implicit conditions (N > 0) can lead to non-obvious non-termination if they are not made explicit.
By encoding data properties in the term structure we can make some implicit conditions explicit and make rule-heads disjoint.
Having numerical conditions in rule-heads would be really nice. Maybe some Prolog implementation in the future allows for that.

Mandelbrot set on EV3 Timelapse

2016-12-08T00:00:00+01:00

I wrote something on a LEGO Mindstorms EV3 that calculates members of the Mandelbrot set. As promised, here is a timelapse of about 120 minutes of calculation. Sorry for the potato quality. The webcam flickers on 60Hz but works fine on 50. The timelapse tool didn't care for framerate-suggestions though.

Clojure and the Mandelbrot set on LEGO Mindstorms EV3

2016-12-05T00:00:00+01:00

The LeJOS operating system for LEGO Mindstorms EV3 is, as I said before, a small Linux with an embedded JVM.

Running Clojure:

To run Clojure on the EV3 I had to download the Clojure runtime, unzip it and throw away everything but the actual jar-file clojure-1.8.0.jar.

Then the jar-file needs to be transfered to the EV3 (via scp for example) and can be executed via ssh with jrun -jar clojure-1.8.0.jar (I already have an alias in my ~/.ssh/config for the EV3). After about two minutes we have a Clojure repl that we can use to answer the important questions of our time.

Takes about two minutes to start the repl

Actually coding something:

The first thing we have to notice is, that the leJOS menu stays visible when we opened the clojure repl. This is a problem insofar, that when we use the Clojure code to interact with (for example) the display, both the leJOS menu and our application want to use the display and their interactions will overlap in unpredictable ways (for me both images were "fighting", but the system might as well crash).

So I created a shell script that kills the existing java process and opens it again once our command ran.

#!/bin/sh
killall java
jrun # ... we will look at that later
~/lejos/bin/startmenu &

Now we want to actually write Clojure code where we calculate members of the Mandelbrot set. The Mandelbrot set is the set of all numbers $c$ for which $z_n$ with $z_0 = 0$ and $z_{n+1} = z_{n}^2 + c$ does not diverge (when $n$ goes to infinity). This is calculated in the complex plane. The numbers for which this doesn't diverge are usually drawn in black while the diverging numbers remain white.

I looked for a Java-based library for the complex numbers and found one with Apache. This library was insofar underwhelming, that taking a complex number to the power of an arbitrary Integer doesn't work as one expects. The library always uses the following equivalence: $y^x = exp(x \times log(y))$ which is fine in general but doesn't work if $y$ is zero, which is the base case but also no problem with positive integer powers. Took me an hour because that is not at all well documented (for the integer-case). So the first thing is, to write our own pow function: (defn pow [y, x] (if (.equals y Complex/ZERO) y (.pow y x)))

Now we can define both $z_n$ and whether any $z_n$ diverges:

(defn zn [n, c] ( if (== n 0) Complex/ZERO (.add c (pow (zn (- n 1) c) 2.0 ) ) ))
(defn diverges [x] (or (.isNaN x) (.isInfinite x) (< 2.0 (.abs x))))

And the idea is, to set the whole display to black and evaluate for every black pixel $z_1$ and then for every remaining black pixel $z_2$. Once a pixel was set to white, we no longer need to evaluate it, because we already know it diverges.

So here's the full code for mb.clj:

(import [org.apache.commons.math3.complex Complex])
(import [lejos.hardware.lcd LCD])
(import [lejos.utility Delay])

(defn pow [y, x] (if (.equals y Complex/ZERO) y (.pow y x) ))
(defn zn [n, c] ( if (== n 0) Complex/ZERO (.add c (pow (zn (- n 1) c) 2.0 ) ) ))
(defn diverges [x] (or (.isNaN x) (.isInfinite x) (< 2.0 (.abs x))))

(defn scale_x [x] ( - (/ (* 2.5 x) LCD/SCREEN_WIDTH ) 2.0 ))
(defn scale_y [y] ( - (/ (* 2.0 y) LCD/SCREEN_HEIGHT) 1.0 ))

(doseq [x (range  LCD/SCREEN_WIDTH )
        y (range  LCD/SCREEN_HEIGHT)] (LCD/setPixel x y 1))

(doseq [rep (range 1 1000)]
  (doseq [x (range  LCD/SCREEN_WIDTH )
          y (range  LCD/SCREEN_HEIGHT)]
          (
              if (== 1 (LCD/getPixel x y))
              (if (diverges (zn rep (Complex. (scale_x x) (scale_y y)))) (LCD/setPixel x y 0))
          )
  )
)

We don't need a delay at the end because this will take long enough.

But now we have to think about running this. Basically we need to include every jar we use into the java classpath and then run the clojure main with our script as the parameter. For org.apache.commons.math3 we need the commons-math3-3.6.1.jar from Apache and for the lejos namespace we need the ev3classes.jar from leJOS (which is not included on the system because it is usually compiled into the finished jar).

Once this is all done we can basically run our application with jrun -cp "../ev3classes.jar:./commons-math3-3.6.1.jar:../clojure-1.8.0.jar" clojure.main mb.clj.

After a few hours, it will look like this

I am pretty sure it is possible to compile the whole jar with the correct main using eclipse/ant and whatnot. But that's the first successful experiment in this direction. Here's a timelapse of the program in action.

Upgrading Fedora 24 to 25

2016-11-24T00:00:00+01:00

Fedora 25 was released a few days ago.

I've upgraded one of my machines. The others are soon to follow. Fedora 25 switches from X11 to Wayland as a default. For my laptop I had to switch last release because my laptop's touchpad was nowhere to be found in X. For this release I am looking forward to not getting to play games on my main desktop. I am sure to blog about any problems I happen upon.

If you're running Fedora 24 you can easily upgrade to Fedora 25 running the following console commands. But before that, make sure your system is fully upgraded and rebooted, especially if you installed some kernel updates.

sudo dnf install dnf-plugin-system-upgrade
sudo dnf system-upgrade download --releasever=25 --allowerasing

The --allowerasing argument allows some packages to be deleted during the process. I had some dangling haskell packages, that needed to be deleted. No problem there.

Once all the downloads are done, type sudo dnf system-upgrade reboot to reboot your system starting the actual upgrading process.

And yeah, I broke my work setup that used proprietary NVidia drivers to get a second monitor running.

Working and Working on LEGO Mindstorms EV3

2016-11-23T00:00:00+01:00

Presentation Space at a Networking Event for Educators

Since October I am working at the Leipzig University of Applied Science as a researcher and project manager in a robotics project. The project is called "Roberta" and is aimed at young people, specifically girls and young women. With consumer-grade robotics sets, like LEGO Mindstorms, we try to get them excited about robotics, computer science and STEM in general. What we're doing is working with educators to get this project to schools.

I am doing project management but I am also doing research and testing the limits of what we can do with those robots. Not from an engineering (or software engineering) perspective. I am not throwing C-code at the machines.

"programming"

The EV3 Bricks (the brain of the LEGO Mindstorms EV3) are running a small Linux. Using the original proprietary LEGO Mindstorms programming software is a bit lacking. You use the graphical interface to create a flow-graph that describes the actions the robot should take. Sadly, this language doesn't support variable scoping (meaning that every variable is global) and only support non-recursive functions. Indirect recursion is prohibited as well (no tricks!). That's no fun.

Alternatively there are multiple (two) alternative Linux-based operating systems. I have yet to try the other one, but the first one is basically a Linux with an embedded JavaVM. This is usually used to, well, program Java. I've tried that and it's boring because Java is boring. But having the JVM is, even with 300Mhz and 64MB RAM, pretty powerful.

I've tried to run Jython on the brick but there wasn't enough memory to do it. Using a USB Stick formatted as swap I was able to get it to run, but it took almost an hour to even start the repl. Running any actual code took prohibitively long.

16GB of additional very slow main memory ought to be enough for everybody

But Clojure also runs on the JVM and this worked quite nicely. The next article will be about me, running Clojure on a LEGO Mindstorms Ev3.

Writing a Master's Thesis

2016-11-11T00:00:00+01:00

Before I write about the topic my master's thesis touched, I wanted to look at how my writing was going. After six months of writing and about a year of research, the thesis ended up clocking in at 18120 words. That's a thousand words more than Shakespeare's Macbeth. Who would've thought.

Here is my writing progress illustrated:

As we can see, we have a slow start with long fits of unproductivity, picking up the pace in late August. But as it turns out, I did it.

Twitchlive Extension on OMG!Ubuntu! and GamingOnLinux

2016-08-21T00:00:00+02:00

On Wednesday the "OMG! Ubuntu!" blog wrote an article "Get Notified When Your Fave Twitch Channels Go Live With These Linux Apps" that also had a writeup on Gaming On Linux.

The Twitchlive Extension I wrote was mentioned in a very positive context.

Twitchlive Extensions

I am happy that the reactions to the extensions are so positive :)

Attending GUADEC 2016

2016-07-25T00:00:00+02:00

This year I have the chance to attend GUADEC, the gnome user and developer conference in Europe. I am very much looking forward to it. This will be my vaccation this year. I surely hope Karlsruhe is up to it.

Fedora 24 issues

2016-07-17T00:00:00+02:00

So I've been running Fedora 24 on my laptop for a bit and its not as nice as I would have hoped.

First of all, Fedora was no longer under the impression that my device had a touchpad, which it most certainly does. This problem was fixed by moving from the GNOME X-session to the Wayland session. With Wayland, Chrome isn't maximized when it starts even though it thinks it is, but thats just a minor issue. the main problem is that with Wayland, my system no longer recognizes any but the native resolution of my display panel. This is, in itself, not a problem. But any attached projector is also only recognized with its native resolution, leading to the inability to clone the display across both the panel and the projector since they no longer have any resolution in common.

So now I can decide between using my touchpad or a projector.

Using the C.H.I.P. flasher for Chrome on Fedora

2016-07-04T00:00:00+02:00

C.H.I.P. Flasher

Flashing the CHIP from Fedora using the Chrome Flashing plugin is only a little bit different from using Ubuntu. We still have to add the user to the dialout group, to allow accessing the device from the user's context (without root).

sudo usermod -a -G dialout $(logname)

If you're doing arduino-programming, you might already be member of the dialout-group. This can be checked with the groups command. If dialout is mentioned, then you can skip this step and also the re-logging.

Then we need to create a file /etc/udev/rules.d/70-allwinner.rules with the following content:

SUBSYSTEM=="usb", ATTRS{idVendor}=="1f3a", ATTRS{idProduct}=="efe8", TAG+="uaccess", SYMLINK+="usb-chip"
SUBSYSTEM=="usb", ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="1010", TAG+="uaccess", SYMLINK+="usb-chip-fastboot"
SUBSYSTEM=="usb", ATTRS{idVendor}=="1f3a", ATTRS{idProduct}=="1010", TAG+="uaccess", SYMLINK+="usb-chip-fastboot"
SUBSYSTEM=="usb", ATTRS{idVendor}=="067b", ATTRS{idProduct}=="2303", TAG+="uaccess", SYMLINK+="usb-serial-adapter"

This links the CHIP to a location that the Flasher can identify. If you're doing other Allwinner-Flashing (I think the OrangePi does have a similar chip), you might run into problems here. But if you allready do, you probably know whether you're affected or not.

Finally, one needs to reload the udev-rules with sudo udevadm control --reload-rules and logout and login again (or reboot), if the dialout group was added this session. And after that, the flasher should work fine.

Upgrading Fedora 23 to 24

2016-07-03T00:00:00+02:00

Fedora 24 was released a few days ago.

I've upgraded one of my machines. The others are soon to follow. One problem I discovered was that the default Gnome-X-Session does no longer support my touchpad, disabling tap-to-click for me. Switching to the Wayland session worked very well and I have yet to hit any glitch.

If you're running Fedora 23 you can easily upgrade to Fedora 24 running the following console commands. But before that, make sure your system is fully upgraded and rebooted, especially if you installed some kernel updates.

sudo dnf install dnf-plugin-system-upgrade
sudo dnf system-upgrade download --releasever=24 --allowerasing

The --allowerasing argument allows some packages to be deleted during the process. I had some dangling haskell packages, that needed to be deleted. No problem there.

Once all the downloads are done, type sudo dnf system-upgrade reboot to reboot your system starting the actual upgrading process.

Newest Family Member is a PocketC.H.I.P.

2016-07-02T00:00:00+02:00

C.H.I.P. and PocketC.H.I.P.

Recently we've had a new addition to our family. A kickstarted one. So in the future, there probably will be relevant articles here as well.

There are allready a few things planned with this 9$ PC.

Teaching again (advanced programming)

2016-05-09T10:00:00+02:00

At the end of this semester I will give six hours worth of exam preparation for the 2nd year computer science bachelor's students in "advanced programming".

While I have yet to consult with the teacher, what the focus of the exam will be, the following topics are covered during the run of the course:

Formulas and terms, signatures, algebraic data structures, tree-domains, pattern-matching, rewriting systems
Higher order functions, polymorphic functions (including typeclasses), lambda-calculus, recursion patterns
lazy evaluation and infinite data structures
git version management as a directed acyclic graph, code smells, refactoring

Most of the time the students try to apply the material with Haskell and C#.

While this course is more structured than the last one I helped preparing for the exam, the examination itself is not that rigid and it will be a fun challenge that I am very much looking forward to.

Installing Cockpit on Raspbian

2016-03-27T00:00:00+01:00

Cockpit

I was playing a bit with with a Raspberry Pi B and raspbian and I wanted to install Cockpit on my pi. Sadly, we can't use the provided debian repository since they do not provide packages for our Pi's processor architecture.

So we do it ourselves.

First thing, we need to install node. If you already have a current node installation, you don't need to do that.

Starting with node, we download the latest node source code and unpack it wget https://nodejs.org/dist/v5.9.1/node-v5.9.1.tar.gz && tar -xzf node-v5.9.1.tar.gz. We now can just cd in there (cd node-v5.9.1) and build and install it (./configure && make && sudo make install). This will take quite some time. This might even take a few hours. After the configure step, you can make > makelog & and then disown in order to leave that session alone and even close it. But don't forget to sudo make install once it's done.

Now on to building Cockpit itself.

First we need to install all the build dependencies:

sudo apt-get install autoconf intltool libglib2.0-dev libsystemd-journal-dev libjson-glib-dev libpolkit-agent-1-dev libkrb5-dev libssh-dev libpam-dev libkeyutils-dev glib-networking

After we've installed all the build dependencies, we can download the sources of the latest release from github. In my case, this was 0.99.

wget https://github.com/cockpit-project/cockpit/archive/0.99.zip && unzip 0.99.zip

After we cd-ed into the source directory, we can, more or less, follow the building instructions.

Create and switch to the build directory and run autogen.sh. I had to disable pcp because I wasn't able to find the header files in the raspbian repository. We also disable documentation creation.

mkdir build
cd build
../autogen.sh --disable-pcp --disable-doc

And once this step is done, we compile it, install it and copy some authorization files.

make
sudo make install
sudo cp ../src/bridge/cockpit.pam.insecure /etc/pam.d/cockpit
sudo sh -c "cat ../src/bridge/sshd-reauthorize.pam >> /etc/pam.d/sshd"

And we're basically done. Start cockpit with sudo systemctl start cockpit.socket and enable it to run on boot with sudo systemctl enable cockpit.socket.

You're now good to go to access cockpit on port 9090 or integrate it in your cockpit landscape.

Installing Rocket Chat with own MongoDB instance on a Raspberry Pi B

2016-03-25T00:00:00+01:00

I was tasked with installing Rocket.Chat on a raspberry pi. Currently I have a PiB and a Pi3 that are not tasked with anything. Since I expect myself having other plans for the Pi3, I chose to try installing Rocket.Chat on the PiB.

First we start with installing raspbian on our pi. The light edition should do it. Do not forget to resize the root partition (using fdisk or parted) if you're using the light edition since it is not so easy to do from inside the raspbian system.

But let's say we have a fresh raspbian installation and start from there.

First we have to install all dependencies. Those are Node.js, MongoDB and graphicsmagick. Graphigsmagick we will install through the repository while we will build node, to get the latest and greatest version, ourselves.

In the meantime, if we have multiple terminal sessions or another ssh session, we can start installing the other stuff. graphicsmagick we can install with sudo apt-get install graphicsmagick. No problem there.

MongoDB is a little bit more complex. They do not supply packages for our system, so we need to compile it ourselves as well. But this needs much more memory than our Pi has, so we need to add some swap memory as well.

I had a spare 1GB usb stick lying around so I formatted this drive to swap (with gparted), plugged it into my Pi and enabled it for swapping with sudo swapon /dev/sda1. With more than 1Gb of swap, we are good to go (from what I've seen, the build process peaks at about 800MB of memory).

Now we need to install scons with sudo apt-get install scons. We download and extract the latest source release from the mongodb download page (in my case, this was 3.2.4) with wget https://fastdl.mongodb.org/src/mongodb-src-r3.2.4.tar.gz && tar -xzf mongodb-src-r3.2.4.tar.gz The build process is not so hard, it just takes time. Once we cd ed into the source folder, we can start building the database server with scons. For me, this took slightly less than 30 hours so putting this one in the background as well and then going for a long walk, then a sleep, and then for a long walk again, might be good idea.

After this is done, we install MongoDB with sudo scons install. This will, again, take a some time.

If you've put all the stuff in the background, you can, once in a while, log in and check with top, whether your Pi is still doing something.

I tried building MongoDB myself, before opting for the packages provided. During this exercise, I learned something, that makes me a bit wary.

MongoDB's build process does not work with the source from github. The configure process fails with a source archive from github while the one from the download page, which should be identical, fails. This is quite an issue. Also, I couldn't find a MongoDB bugtracker linked anywhere from github or the MongoDB homepage. This leads me to believe that MongoDB is not really interested in the open source community and fostering an open infrastructure. Rocket.Chat is built upon MongoDB but I won't be using MongoDB for any of my projects.

Solving Tatham's Puzzles - Signpost (backtracking)

2016-03-21T22:15:00+01:00

I've started writing solvers for games of Simon Tatham's Portable Puzzle Collection. Those Problems can usually be reduced to some NP-complete problem. This is quite a fun exercise.

Unsolved Signpost Puzzle

We will be continuing with the puzzle "Signpost", which is a variant of Hamiltonian paths in a directed graph where, for some fields or nodes, the position within the path is given.

The goal of the puzzle is to connect the Node "1" to some node, which is then node "2" in the direction the arrow points to. All numbers need to be connected to their successor until all nodes have a single unique number from "1" to (in this example) "25".

We will be solving this problem in two different ways, both with the help of the programming language Python. But first of all we'll need to convert a puzzle into a machine readable format. Luckily, Signpost has an easy and accessible format. The problem instance from the image is the instance 5x5:1cceefcfggeeccghcac3e12hch10ah25a. It starts off with the size of the problem and then a list of letters that show the directions of the arrow where a is north or 12 o'clock and all other letters go clockwise with h being north-west. A letter can be prefixed by a number which is the hint for that node. That node's number is fixed. The cells or nodes are defined line by line from the top left to the bottom right.

Problems don't need to start with 1 or end with the last number. The start and the end of the path can be anywhere but puzzles generated with the ends at opposite corner look nicer.

First we define what our data looks like. We implement a simple form of directed graph that we define as a set of nodes and their successors. From that we derive following definition:

class Node(object):
    hint = None
    following = frozenset()

    def __init__(self, direction):
        self.direction = direction

We don't want to use any of that code in a library or something something so we don't do anything fancy here and we use the simplest class definition that comes to mind.

^(?P<size_x>\d+)x(?P<size_y>\d+)$

(?P<defn>\d*[a-h])

Now we go on to parsing the problem. The parsing function should get the problem as a string and return a 3-tuple containing width and height of the problem as well as a mapping of positions within the grid to ``Node`` instances.

Onto actually parsing. We can split at the colon and then regular expressions to the rescue. The size definition has the form ^(?P<size_x>\d+)x(?P<size_y>\d+)$ while a single node definition has the form (?P<defn>\d*[a-h]). As we can see, the digits are optional but we need to greedily accept them. All non-overlapping occurences in order will be our cell definitions.

In code we, again, do nothing too too fancy:

def parse(puzzle_str):
    directions = {
        'a': (0,-1), 'b': (1,-1), 'c': (1,0), 'd': (1,1),
        'e': (0,1),  'f': (-1,1), 'g': (-1,0),'h': (-1,-1)
    }

    size, _, definition = puzzle_str.partition(':')
    r_size = re.compile(r"^(?P<size_x>\d+)x(?P<size_y>\d+)$")
    r_defn = re.compile(r"(?P<defn>\d*[a-h])")
    size_t = tuple(map(int, r_size.match(size).groups()))
    w, h = size_t
    nodes = {}
    for n, m in enumerate(r_defn.finditer(definition)):
        pos = (n % w, n // w)
        nodes[pos] = Node(directions[m.group(0)[-1:]])
        hint = m.group(0)[:-1]
        if hint:
            nodes[pos].hint = int(hint)
    return w, h, nodes

And we also need to fill in the successor Nodes. This is no problem at all.

w, h, nodes = parse(sys.argv[1])
from itertools import product
for x, y in product(range(w), range(h)):
    this_node = nodes[(x,y)]
    dx, dy = this_node.direction
    x, y = x + dx, y + dy
    while x >= 0 and x < w and y >= 0 and y < h:
        this_node.following = this_node.following | frozenset([nodes[x,y]])
        x, y = x + dx, y + dy

And now on to actually solving this thing. The idea is, that we slowly build a list of Nodes that is our solution. The first element is the Node with the hint "1" and we iterate through all nodes that follow that Node and do not have a hint that is other than "2", and so on. And of course, the Node that we add to the solution is not allowed to have already been part of the solution and if there is a node with that hint, it should be this exact one. If we find such a conflict in our solution, we reject that solution and return to an earlier partial solution.

Instead of using recursion, we build a queue of partial solutions until we get a complete solution. This is functionally identical to restarting a solver function with partial solutions of increasing size. In that case, the call stack manages the backtracking. But we'll do it with a queue this time. We build the queue so that every element in the list is either

a partial solution (beginning from 1) without conflicts
a partial solution (beginning from 1) where the last element is in some conflict
a complete solution

For case 1, we remove the partial solution that has the length n and add all partial solutions of length n+1 by iterating through all successor Nodes that have not yet been part of the partial solution. For case 2, we reject that partial solution. For case 3, we immediately return that solution and are finished.

def solve_dhc(nodes):
    # get the Node that has hint 1
    first = next(iter(n for n in nodes.values() if n.hint == 1))
    # get the set of all hints, so it is easier to check for conflicts
    hints = frozenset(n.hint for n in nodes.values() if n.hint is not None)
    from collections import deque
    q = deque([[first]]) # initializing queue
    while q:
        curr = q.pop()
        idx = len(curr)
        if (idx in hints or curr[-1].hint is not None) and curr[-1].hint != idx:
            continue # case 2
        if idx == len(nodes):
            return curr # case 3
        for _next in curr[-1].following: # case 1
            if _next not in curr:
                q.append(curr + [_next])

This algorithm terminates because we always remove one element from the queue of some length n and possibly add a large but finite amount of elements of length n+1 and the algorithm terminates when we have an element in the queue of some possibly large but finite length.

Sidenote: if you use popleft instead of pop you get a breadth-first search instead the depth-first search, that is implemented here. Add to the right and pop from the right for depth-first and add to the right and pop from the left for breadth-first. Since every proper solution has the exact same length/depth (that we know and never go further anyways), searching depth-first is strictly better than breadth-first.

And given a solution, we still have to print it. That's the easy part.

dhc = solve_dhc(nodes)

fill = len(str(w*h))
for l in range(h):
    for c in range(w):
        node = nodes[(c,l)]
        print(str(dhc.index(node) + 1).rjust(fill), end=" ")
    print("")

Solved Puzzle

And there's the solution for our puzzle.

$ python3 signpost_dhc.py 5x5:1cceefcfggeeccghcac3e12hch10ah25a
 1 20  9  2 21
23 14 13 22 24
15  5  7  6  8
18 19 11  3 12
16 17 10  4 25

My computer science course of Jan 2016

2016-02-11T18:40:00+01:00

As I told before, in January I have been teaching a class preparing for an exam in theoretical computer science master's students.

There were little over 20 people in attendance and it was a nice experience I wouldn't mind repeating. During the evaluation process (I hope to get access to the official evaluation dataset) I got almost only high marks in all categories which I'm rather proud of.

I was a bit baffled when I asked for a list of well-known set operations and stared into blank faces but one never knows if it is shyness or missing knowledge keeping the answer from being given. I'd like it to be the former.

Since, after my first teaching job, I've yet to run out of hope, I offered to answer questions via email after the fact and a few messages laster, here is what most students seem to have problems with:

How do you find out whether a language is regular/context-free?

For both types of language we have something called the "pumping lemma" which is basically that in any infinite language with a finite set of production rules, there exists a list of words that form a repeating pattern that has a relation to a loop within the production rules.

For regular languages (say $R$) it is basically $L \in R \to (\exists uvw \in L: uv^*w \subseteq L)$ with the condition of $v$ being not empty. The idea is, that a regular language can be represented as a deterministic finite automata (DFA) that has a finite number of states while the language is infinite so that some state has to be reached more than once in the course of accepting a word. After reading $u$ you are in some state and then you read $v$ and are in the same state you were after reading $u$ and after then reading $w$ the word is accepted. The word $v$ can be repeated many times or not at all.

CC BY-SA 3.0 / Jochen Burghardt

For context free languages (say $CF$) this is nearly the same, only that there is some nonterminal that can produce itself from the production rules. Repeating words appear left and right of the looping nonterminal equally often ($v$ and $x$). The looping nonterminal also produces a word without looping (or every production including that nonterminal wouldn't halt) that we call $w$. So the formula for the context-free version of the pumping lemma is basically $L \in CF \to (\exists uvwxy \in L: (\forall n \in N : uv^nwx^ny \in L))$ with the sidecondition of not both $vw$ and $wx$ being empty.

In both formulas have the form $L \in L' \to PL$ and we get the reversal $\neg PL \to L \notin L'$ for free. So to show that a language is regular or context free it is not enough to show that the pumping lemma holds because there are languages that are not in that category and still have words that can be pumped.

So to show that a language is context free/regular one has to construct a context free grammar/(non)deterministic finite automata and if you can't find any way to pump words then the language is not in that category.

The direction of the implication-arrow (or that it isn't an equivalence) seems to have my students stumped.

And here you go with two languages that fulfill the pumping lemma for a language category and are not a language of that type:

$a^n | n \text{ is not prime}$ with $(u,v,w) = (aa,aa,\varepsilon)$
$a^nb^n | n \text{ not a power of two}$ with $(u,v,w,x,y) = (\varepsilon,aaa,aabb,bbb,\varepsilon)$

Solving Tatham's Puzzles - Pattern

2016-01-31T19:07:00+01:00

I've started writing solvers for games of Simon Tatham's Portable Puzzle Collection. Those Problems can usually be reduced to some NP-complete problem. This is quite a fun exercise.

We will be starting with the puzzle "Pattern", also known as Nonograms that we reduce to SAT.

Unsolved Nonogram

First to explain the puzzle: The cells of the puzzle can be filled with black or white squares. The numbers on the sides of the rows and colums denote the number of consecutive black squares. Every run of black squares has to be seperated from another run by white squares.

First we want to define a type for our problem.

type PSize = (Int, Int)
type Run = [[Int]]
type Runs = (Run, Run)
data Problem = Problem PSize Runs deriving Show

The puzzle on the side is defined in the Tatham format by the string 10x10:2/3.2/1.1.3/2.5/2.2.4/1.1.4/1.3/1.1.2/4/4.1/5.2/2.3.3/1.2/6/3/1/4/6/7/4.1. It has a size information and then the list of runs. Another format we want to support is the Haskell Runs format which would be ([[2],[3,2],[1,1,3],[2,5],[2,2,4],[1,1,4],[1,3],[1,1,2],[4],[4,1]],[[5,2],[2,3,3],[1,2],[6],[3],[1],[4],[6],[7],[4,1]]). This format seperates the runs for rows and columns and ommits the size information.

For the second format we can use Haskell's own read function. For Tatham's format we have to define a parser. I've added type information for most functions which is often needed for code involving read.

import Text.ParserCombinators.ReadP
number' = many1 (choice (map char ['0'..'9']))

r_size :: ReadP PSize
r_size = do l <- number'
            char 'x'
            r <- number'
            return (read l, read r)

r_run :: ReadP [Int]
r_run = do { li <- sepBy number' (char '.'); return $ map read li }

r_runs :: ReadP [[Int]]
r_runs = sepBy r_run (char '/')

r_problem :: ReadP Problem
r_problem = do size <- r_size; char ':'
               runs <- r_runs
               skipSpaces; eof
               return $ Problem size (take (fst size) runs, drop (fst size) runs)

What do we now do with such a problem? First we want to check it for consistency. If we got size information it should fit the runs and the numbers in the runs (and the spaces between them) shouldn't add up to more than the size.

import qualified Prelude as P (all, (&&), not)
consistent :: Problem -> Bool
consistent (Problem size (xin, yin)) = consistent' size xin P.&& consistent' (swap size) yin
consistent' (w, h) xs = (w == length xs) P.&& P.all (\x -> (sum x) + (length x - 1) <= h) xs

Calling our solver

But now on to solving the problem. We will reduce this problem to SAT using the Ersatz library. The encoding of a problem, I guess, is obvious. We say that every cell within the Nonogram is a Variable Bit and a bit being True has the semantics of that field being colored black while False would mean white.

We want to build constraints of single rows or columns and then solve the conjunction of all those row/column-constraints for satisfiability. Therefore we define a function of the type [Int] -> [Bit] -> Bit that takes a list of Integers (the numbers of one column or row) and the list of remaining Bits that do not yet are constrained.

First we begin with the end of the recursion. If no number is left or the remaining number is zero (for empty columns or rows), we want all remaining variables to be False. (we take for to be defined as flip map)

build_runs :: [Int] -> [Bit] -> Bit
build_runs nums vars = case nums of
  [] -> all not vars
  [0] -> all not vars
  x:xs -> let max_d = length vars - (length xs + sum xs) - x
              run   = [not] ++ replicate x id ++ [not]
              v     = [false] ++ vars ++ [false]
          in any id $ for [0..max_d] $ \ d ->
             (all not $ take d vars)
             && (all id $ zipWith id run $ take (x + 2) $ drop d v)
             && (build_runs xs $ drop (x + d + 1) vars)

Solved Nonogram

The recursion is something to think about a bit. We're given a number of consecutive black squares x and all the variables in the column/row vars. The x Trues can be put anywhere from 0 to max_d where max_d is all the rest of the variables take the minimum necessary space of the remaining number of numbers xs, while all variables before that number of black squares and one after need to be white. For every possible position the rest of the variables need to fulfill the same condition with the rest of the numbers.

Now we have to combine all the constraints from a given problem.

nono :: (MonadState s m, HasSAT s) => Problem -> m (Map (Int,Int) Bit)
nono (Problem (sizex, sizey) (xin, yin)) = do
  let indices = [(x, y) | x <- [1..sizex], y <- [1..sizey] ]
  vars' <- replicateM (length indices) exists
  vars <- return $ Map.fromList $ zip indices vars'
  let getx x = for [1..sizey] $ \y -> (vars ! (x,y))
      gety y = for [1..sizex] $ \x -> (vars ! (x,y))
      xs = zipWith build_runs xin (map getx [1..])
      ys = zipWith build_runs yin (map gety [1..])
  assert (all id xs && all id ys)
  return vars

Internally we let Ersatz identify variables by their position in the grid Map (Int,Int) Bit. Now we can solve our problem easily using minisat with Ersatz's native minisat binding. Given a solution, we just format it correctly and give it to the user.

main = do
  input <- getContents
  let parsed = parse_tat input
      problem = case parsed of
        Just p -> p
        Nothing -> let (xin, yin) = read input :: ([[Int]],[[Int]])
                   in Problem (length xin, length yin) (xin, yin)

  putStrLn $ "Size: " ++ (show $ problem_size problem)
  unless (consistent problem) $ do {putStrLn "Problem inconsitent"; exitFailure}

  (Satisfied, Just solution) <- solveWith minisat (nono problem)
  let sizey = snd $ problem_size problem
      lines = for [1..sizey] $ \i -> filter (\ ((x, y), b) -> y == i) $ toList solution
      conv  = map $ \((x,y), b)-> if b then 'X' else ' '
      join = foldr (:) ""
  forM_ (map (join . conv) lines) print

And we're done.

Gnome Foundation membership renewed

2016-01-25T14:20:00+01:00

My membership in the Gnome Foundation has been renewed. I've been co-maintaining several of the included games for the last two-and-a-half years. My membership has now been renewed until early 2018. I hope I can find a little more time to spend on our favorite desktop environment in the near future.

wordpress certificate error preventing upgrade

2015-12-09T13:04:00+01:00

I had the following problem after upgrading my wordpress installation to 4.4: SSL certificate problem, verify that the CA cert is OK. Details: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed.

The problem is that wordpress ships its own, old certificate bundle. You can fix this by downloading http://curl.haxx.se/ca/cacert.pem and overwrite your /wp-includes/certificates/ca-bundle.crt with it.

This will be needed after every wordpress upgrade that does not fix the issue itself which is probably an outdated server missing the certificate or a php version being compiled with an old version of openSSL.

Teaching theoretical computer science in January

2015-12-09T12:26:00+01:00

In January I will give six hours worth of exam preparation for this year's theoretical computer science master's students.

We will cover the following topics:

Deterministic and Nondeterministic Finite Automata, reduction from NFA to DFA and pumping lemma
Context-free and Context-sensitive grammars and corresponding languages and pumping lemma as well as the word problem in Context-free grammars
Basic complexity theory (P, NP, NP-Complete), decidability and reductions between several NP-Complete problems

I am looking forward to it.

Haskell/DG

2015-11-13T23:00:00+01:00

Haskell	Dogelang
import System.Environment import Data.String import Data.List import Data.Char import Data.Ord data Tree a b = Node a b (Tree a b) (Tree a b) \| Nil deriving (Show) letters = "abcdefghijklmnopqrstuvwxyz" filter_letters = filter (flip elem letters) filter_words = filter (((<) 2) . length ) update t k def fn = case t of Node k' v' cl cr -> case compare k k' of EQ -> Node k' (fn v') cl cr LT -> Node k' v' (update cl k def fn) cr GT -> Node k' v' cl (update cr k def fn) Nil -> Node k def Nil Nil inorder t = case t of Node k v cl cr -> (inorder cl) ++ [(k, v)] ++ (inorder cr) Nil -> [] main = do args <- getArgs content <- readFile (args !! 0) let ws = filter_words $ map filter_letters $ words $ map toLower content t = foldl (\t w -> update t w 1 ((+) 1)) Nil ws print $ take 20 $ inorder t print $ take 20 $ sortBy (\a b -> compare (snd b) (snd a) ) $ inorder t	import "/sys/argv" import "/collections/namedtuple" import "/string/ascii_lowercase" Node = namedtuple "Node" ("k", "v", "cl", "cr" ) letters = ascii_lowercase filter_letters = (bind str.join "") <- bind filter (in letters) filter_words = bind filter ((> 2) <- len) update = t k def fn -> if (k', v', cl, cr = t) => if (k == k') => Node k' (fn v') cl cr (k < k') => Node k' v' (update cl k def fn) cr (k > k') => Node k' v' cl (update cr k def fn) t is None => Node k def None None inorder = t -> if (k, v, cl, cr = t) => (inorder cl) + [(k, v)] + (inorder cr) t is None => [] main = -> content = open $ argv !! 1 \|>.read! words = (bind str.split sep:" ") <- (bind str.join " ") <- str.splitlines ws = filter_words $ map filter_letters $ words $ str.lower content t = foldl (t w -> update t w 1 (+ 1)) None ws print $ list $ take 20 $ inorder t print $ list $ take 20 $ sorted (inorder t) key:snd reverse:True main!

Haskell

Dogelang

import System.Environment
import Data.String
import Data.List
import Data.Char
import Data.Ord

data Tree a b = Node a b (Tree a b) (Tree a b) | Nil deriving (Show)

letters = "abcdefghijklmnopqrstuvwxyz"
filter_letters = filter (flip elem letters)
filter_words = filter (((<) 2) . length )

update t k def fn = case t of
  Node k' v' cl cr -> case compare k k' of
                      EQ -> Node k' (fn v') cl cr
                      LT -> Node k' v' (update cl k def fn) cr
                      GT -> Node k' v' cl (update cr k def fn)
  Nil -> Node k def Nil Nil

inorder t = case t of
  Node k v cl cr -> (inorder cl) ++ [(k, v)] ++ (inorder cr)
  Nil -> []

main = do
  args <- getArgs
  content <- readFile (args !! 0)

  let ws = filter_words $ map filter_letters $ words $ map toLower content
      t = foldl (\t w -> update t w 1 ((+) 1)) Nil ws
  print $ take 20 $ inorder t
  print $ take 20 $ sortBy (\a b -> compare (snd b) (snd a) ) $ inorder t

import "/sys/argv"
import "/collections/namedtuple"
import "/string/ascii_lowercase"



Node = namedtuple "Node" ("k", "v", "cl", "cr" )

letters = ascii_lowercase
filter_letters = (bind str.join "") <- bind filter (in letters)
filter_words = bind filter ((> 2) <- len)

update = t k def fn -> if
  (k', v', cl, cr = t) => if
              (k == k') => Node k' (fn v') cl cr
              (k < k')  => Node k' v' (update cl k def fn) cr
              (k > k')  => Node k' v' cl (update cr k def fn)
  t is None => Node k def None None

inorder = t -> if
  (k, v, cl, cr = t) => (inorder cl) + [(k, v)] + (inorder cr)
  t is None => []

main =  ->

  content = open $ argv !! 1 |>.read!
  words = (bind str.split sep:" ") <- (bind str.join " ") <- str.splitlines
  ws = filter_words $ map filter_letters $ words $ str.lower content
  t = foldl (t w -> update t w 1 (+ 1)) None ws
  print $ list $ take 20 $ inorder t
  print $ list $ take 20 $ sorted (inorder t) key:snd reverse:True

main!

Fpp

talk at Leipzig's Haskell conference HaL-10

2015-11-12T17:25:00+01:00

As I said before: I will hold a small talk about dogelang at Leipzig's next Haskell conference HaL-10.

Here's a translation of the abstract (original in German):

dg - The better Haskell is named Python

Who needs typeclasses? Or static type system even? Why have side-effects to be so complicated? And what are monads?

There must be something better!

dg is Python with Haskell syntax. In a short talk (15 minutes) I want to introduce, on a funny note, the "better Haskell".

Under the slogan "what we don't have, you won't need" we will be looking at features of the Haskell language and their counterparts in dg. To conclude, we will take a look at how intelligent a Python program can look, if it has the proper syntax.

Deciding on the pure-ness of a python function

2015-11-12T17:21:00+01:00

Through the work with dogelang I am currently looking at pureness of python functions and on how to decide and annotate pureness so that I, in theory, could write a static checker that checks whether the annotated functions are pure.

We can take the definition of pureness from the wikipedia and it is as follows

The function always evaluates the same result value given the same argument value(s). The function result value cannot depend on any hidden information or state that may change while program execution proceeds or between different executions of the program, nor can it depend on any external input from I/O devices.

Evaluation of the result does not cause any semantically observable side effect or output, such as mutation of mutable objects or output to I/O devices.

Python makes few restrictions on side-effects and mutability and since we even can override basic operators (like the dot-operator for attribute access), it is hard to infer anything remotely useful or non-trivial from a python-program.

I have been, for example, responsible for code like that (paracoded for your convenience):

def cls(object):
  def __init__(self):
    self.d = {}

  def __getattr__(self, attr)
    if not attr in self.d:
      self.d[attr] = someobj(attr)
    return self.d[attr]

A = cls()
A.foo # this line has side effects

This violates the basic principle of python that an expression should either have a return value or a side-effect but not both. But we might have done this for good reason. In this case, the academic value of the "front-end" of the library outweighs the atrocities of the implementation. But it serves as a fine example that, it is hard to reason about even simple looking expressions, if they include the dot-operator (or any other operator for that matter), especially if you don't have the full class definition at hand.

But it isn't only user-made classes that don't hold true to that principle. Most low-level functions that interact with the operating system do that in some way. They have their side-effect within the realms of the rest of the system and still they have a return value that represents some aspect of the operation within our program.

os.write(fd, str)

Write the bytestring in str to file descriptor fd. Return the number of bytes actually written.

open(file, ...)

Open file and return a corresponding file object. If the file cannot be opened, an OSError is raised.

So we would want to mark any function as impure that has a reference to one of these impure functions. But we would also want to mark any functions defined within these impure functions to be themselves impure. Looking at the following code it would be hard to infere whether a function has a reference to, let's say 'open'.

o, f = open, "file.tmp"
def fun():
  o(f)

Given Rice's theorem, I'd think that properly tracking references would either be undecidable or at least a problem of exponential growth (tracking all possible assignments to some reference). So to be on the safe side, we state the following as our impurity rule number 1: A function is impure if it uses a reference from an impure function or has a reference to an impure function. We would also want an exhaustive list of builtin impure functions, applying the rules recursively.

In the former code example the surrounding block leaks (that's what we will call it) the reference to both o and f into the namespace of fun. This would not be a problem. We even want this behaviour. Let's look at this example:

def f():
  return x*x
x = 5
print(f()) # 25
x = 7
print(f()) # 49

This is contrary to our definition of purity, that the function, given the same arguments should evaluate to the same value. So this only happened, because the reference was updated. Let's add impurity rule number 2: A function is impure if a name is assigned twice and a reference to that name is leaked. This should cover all cases since shenanigans like the following cause a NameError.

def a():
  yield (lambda : x*x)
  x = 3
  yield None
x, i = 5, a()
g = next(i)
print(g()) # Name Error: free variable 'x' referenced before assignment in enclosing scope
next(i)
print(g())

Here we see the yield-keyword we have the case of suspended computation. The next-function is of course part of the set of impure builtin functions because it returns a value and advances the internal state of the generator. Since for rule 2 we introduced the leaked name, we can still allow multiple assignments within a function if the name is not leaked because we want to allow, for example, the generator-function infinity to be pure.

def infinity():
  i = 0
  while True:
    yield i
    i = i + 1

Here, I find, we arrive at an impasse. Every iteration over a generator "uses the generator up". We could say that we won't allow references to generator-objects but that seems arbitrarily strict. Also, we can't really know which name could be used by a generator-object which would lead to not allowing any bare names (only function calls). Disallowing generators seems impossible since map and filter and other functional primitives depend on it.

I would welcome any ideas for a rule that would solve this issue of leaking the internal state of a generator.

CoffeeScript is a missed opportunity

2015-11-09T23:00:00+01:00

list comprehension
missing primitves
associativity of application
missing bind
prefixing/infixing operators/functions

Musings on Graph Isomorphism pre-results

2015-11-05T20:30:00+01:00

I think by now everybody has read that Babai et al will hold a seminar next week where they want to "outline an algorithm that solves the Graph Isomorphism problem [...] in quasipolynomial time".

So they are shooting for a result with the complexity $2^{O(log(n))^c}$ while the best former result (by Luks, 1983) for Graph Isomorphism was $2^{O(\sqrt{n*log(n)})}$.

What is Graph Isomorphism? Given two graphs, the question is, whether there is a mapping function $f$ between the nodes of one graph $G_1$ to the nodes of the other graph $G_2$ so that every edge $(a,b) \in G_1$ is also an edge $(f(a),f(b)) \in G_2$.

So why is this quasipolynomial when $2^x$ is obviously not a polynome? That's where the quasi comes from. If the $c$ in $2^{O(log(n))^c}$ is $1$ then this simplifies to $2^{O(log(n))}$ which is a polynomial. Remember that the base of both the logarithm and the exponentiation isn't actually $2$ so this doesn't simplify to $O(n)$ but some other polynomial (but a polynomial nevertheless). And the $c$ is just a factor in the formula (like the base the logarithm and the exponentiation) that have no impact on the complexity properties (the general shape of the formula describing the complexity) but can be changed by optimization (opposite to groundbreaking research). So given proper optimization the algorithm we will be presented could be (theoretically) as fast as a polynomial algorithm.

When we talk about "fast" we talk about how the runtime changes (relatively) when the problem size grows larger. A "faster" algorithm has smaller relative growth while it might still be orders of magnitude slower in actual runtime for any practically sized input.

Is this result surprising? Well, yes and no. On the one hand the "witness" (or positive solution to the problem) is only of linear size (one mapping for every node in the graph) and the algorithm for checking would be basically quadratic in complexity (checking the definition given above for all edges and there are only quadratically many). This holds for problems in NP-complete but also for problems in P. I think it is obvious that this answer would not be very hard to find (we surely don't need to check every possible mapping - this would basically be the case for NP-complete) but it isn't obvious that the answer is easy to find either (like for a problem in P).

So what we're probably going to see is a clever application of some result of group theory that is applicable here and excludes a large number of all possible isomorphism as possible solutions. So the problem is still not easy but math can help a lot.

What does it mean? It is not known whether Graph Isomorphism is NP-Complete. This result would be a strong hint, that it isn't (albeit no proof). Even if we found Graph Isomorphism to be in P, this would have no larger impact on the field of complexity theory in general.

What's the next thing? That would be finding an algorithm properly in P. If such an algorithm exists, we would also find a way to compute a small "witness" (or in this case "proof trail") that is an easily checkable answer to the question "is there no such isomorphism?". How this proof trail would look is even less obvious. There could be some function that, given a graph as input calculates a value and all graphs that have an isomorphism between them have the same value (which would also need to not be exponential in size). In practice it is very difficult to even find hard instances of Graph Isomorphisms. So it might turn out that there are actually only easy cases, we just don't know it yet.

dogelang adding pattern matching

2015-11-05T12:40:00+01:00

If you haven't read about dogelang, you really should. It is python with haskell-syntax where you can use all python modules. So basically coffee-script but for a language that is already useful. And python bytecode is emitted instead of first compiling to python itself, but that just as a side-node.

On my behest, the developer of dg added pattern matching so that in an if-condition a tuple-unpacking-expression does not raise an exception if the unpacking fails but returns false.

Therefore one can now implement their own zip and list-constructor in the following manner:

zip = x' y' -> if
  ([x, *xs], [y, *ys] = x', y') =>
    yield (x, y)
    yield from $ zip xs ys
  otherwise => yield from ()

list = x' -> if
  ([] = x') => []
  ([x, *xs] = x') => [x] + list xs

This looks almost like haskell's case-pattern-matching but not quite, so it is sure to annoy every haskell-programmer.

You might need to execute python3 -m dg -b once after installing/updating dg to rebuild the interpreter/compiler-bundle with this feature.

In other news: It is quite likely that I will hold a small talk about dogelang at Leipzig's next Haskell conference HaL-10.

Upgrading Fedora 22 to 23

2015-11-05T11:20:00+01:00

Fedora 23 was released a few days ago.

I've upgraded one of my machines. The others are soon to follow. I have yet to hit a problem with Fedora 23.

If you're running Fedora 22 you can easily upgrade to Fedora 23 running the following console commands. But before that, make sure your system is fully upgraded and rebooted, especially if you installed some kernel updates.

sudo dnf install dnf-plugin-system-upgrade
sudo dnf system-upgrade download --releasever=23 --best

The --best argument makes sure if there are any transaction problems, the installation will not continue. I've had to remove some old gstreamer-plugins that are sure to come back with a new installation of fedy.

Once all the downloads are done, type sudo dnf system-upgrade reboot to reboot your system starting the actual upgrading process.

technically, Hello Internet

2015-11-04T15:20:00+01:00

This is my new blog housing all the technical computer-science and programming stuff that doesn't fit nicely within my wordpress installation. I will also post links and the usual "to-remind-me" stuff.

This page will be in English where applicable.