Python list comprehension: so close

Perl has some very nice list management features, but Python's list comprehension is clearly a more powerful approach for simple list transformation. For example, in Perl, when you want to get the list of the "apple" key from a list of hashes:

  @apples = map { $_->{apple} } @fruitstuff;

In Python that looks so much cleaner as a list comprehension:

  apples = [ d['apple'] for d in fruitstuff ]

However, Python starts to get messy when you begin to contemplate pruning the result. Again, the Perl for trimming out undefined values:

  @apples = grep { defined $_ }
            map { $_->{apple} } @fruitstuff;

and the Python:

  apples = [ d['apple'] for d in fruitstuff
             if d['apple'] is not None ]

That's one way to do it, but really, it's not what the Perl is doing. This code is re-indexing fruitstuff to get the value after first indexing it to check that it's defined. The Perl, on the other hand is indexing the hash only once, getting all mapped values and then trimming out the undefined entries. In this example, either approach might work, but it's significant in some cases where indexing might be expensive or have side-effects. So, what does the actually equivalent Python look like?

  apples = [ x for x in [ d['apple'] for d in fruitstuff ]
                   if x is not None ]

Now, functionally these two approaches are identical, but when I approach the Perl, I get a certain amount of clear-headedness by observing that the first function has a name ("grep") a block, functor, closure or whatever you want to call it, and then a parameter which is, itself a function ("map") a block and a list.

When I look at the Python, I have to unpeel it visually and transition in and out of the inner block. Nesting the two list comprehensions makes them quite difficult to read because of the postfix logic operator.

Python's list comprehensions are superior in most ways to Perl's handling, mind you, and I would be doing a disservice to Python to suggest otherwise. But, it's these minor sticking points where I have to wonder why it is that Python chooses to be an almost-functional (in the computer science sense) language. Sure, you can use Python's limited lambda here with its map function, but it doesn't gain you anything, since you still need list comprehension for the reduction, so it will just end up complicating the inner loop (note that in Perl, you can use a single map statement to do all of the work, since map can reduced the size of the returned list, but that wasn't the point of the example).

I like both Perl and Python. They both have their places in my toolbox, but I'll always long for a language that can be all of the promise of both languages without the warts of either...

Ideally, we would take Python's ability to name and scope the iterator variable (not to be confused with the iterable parameter) with perl's chained "Schwartzian transforms").

In pseudo-code this would be:

  resultlist = filter variable: functor, input_iterable

So, in Python, something like:

  apples = filter x: x is not None, \
     [ d['apple'] for d in fruitstuff ]

where filter has pretty much exactly the same syntax as lambda, but with an additional, positional parameter after the body.

In Perl, we would need to add the named iterator variable:

  @apples = grep $x { defined $x }
     map $d { $d{apple} } @fruitstuff

Much cleaner on both counts, but of course, the two communities are too busy treating each other like Baptists and Lutherans at a PTA meeting to see the value in each other's approaches...