Strangely Consistent

Musings about programming, Perl 6, and programming Perl 6

Feedback on "Macros: Placeholdeeers!"

These comments pertain to this pooost.

Macros: Placeholdeeers!

Alternative blog post title: finally, a rant about the {{{ }}} syntax.

The macro blog post train lumbers on. Remember, the goal here is to view macros from all perspectives, or at least all of them that somehow pertain to Perl 6. There's a whole lotta perspectives! Together they will form a "cloud" of suggestions, forces, and decisions... and then, magically, something wildly appropriate will crystallize out of the chaos, suggesting a way forward.

In the meantime, it will seem like I'm recklessly firing RFCs on myself until I'm buried under them. What can I say? You need a lot of different parts to build a... SPACESHIP!

Today's episode brings you vanviegen's macro proposal from our friends over at the CoffeeScript world. You should read the OP in that one, but as a public service, I will also translate allsome of the Coffee code snippets into Highly Plausible Perl 6.

# optionally compiled debug info
macro debug(*@args) {
    if (DEBUGGING) {            # it's a constant defined somewhere
        Q::Call.new(
            Q::Literal.new(     # (handwave)
                "debugImpl"
            )
        )
    }
}

vanviegen's proposal doesn't have a notion of quasi blocks. At first I thought that was pretty lame, and clearly not a step up from what we already have implemented. But then...

# mandatory swap example
macro swap($a, $b) {
    Q.codeToNode({
        my ($x, $y) = $y, $x;
    }).subst(
        '$x' => $a,         # o.O
        '$y' => $b,
    );
}

...uuummmm.

Hold on just a minute. Let's all freeze our immediate reaction, and render that in Current Actual Perl 6:

# mandatory swap example
macro swap($a, $b) {
    quasi {
        ({{{$a}}}, {{{$b}}}) = {{{$b}}}, {{{$a}}};
    }
}

(Which works, by the way.)

Ok, so here's my reaction:

Ok, enough making fun of our own choices. I think what we're seeing here is an interesting solution, but I don't think we should rush to adopt it wholesale. I'll try to be more clear, or at least less circumscript, in what follows.

But I also think I've figured out something about templating syntax.

All templating syntax sucks.
    — masak's Deplorable Truth About Templating Syntax

It doesn't matter what you do: your templating syntax is going to suck. The template placeholders are a kind of out-of-band signaling, OK. "Hey hey, a thing gets inserted here!" You need a syntax for the placeholders. The syntax cannot blend in too naturally into the surrounding text because (a) if people don't notice them then we have a problem, and (b) if they're too close to normal syntax then there are also parsing collisions.

So you have to make them stand out, which means making them look somehow unnatural. Visual pills. Ugly. I guess that's what we went for with the {{{ }}} syntax: maximal ugly. Something that would never, ever be confused with sane and decent code. Someone might at some point find a reason to write two opening braces in normal code. But three? So it's collision-safe. And ugly.

Don't believe me when I say that all templating syntax sucks? Fine, I'll bring you examples:

Anyway, you can't go right with a templating system. It's like the Prime Directive in Star Trek. The string would be better off if you left it in its natural state, but for one reason or another, you just have to inject those out-of-band placeholders and mess things up a bit. Just a little, because things will be "better". After which... things are not in a natural state any more. Your string is full of percents, your HTML no longer validates, your Python has braces, and your Perl 6 has a lot of braces.

But v's macro system offers an interesting alternative. No quasi quotes, no unquoting. Nothing needs to be out-of-band anymore. Things are injected from the outside.

I'm going to list what I perceive as pros and cons of this proposal. In increasing complexity order, so pros and cons will interleave.

Pro: No more {{{ }}}. Yay!

Pro: No more quasi blocks. Actually, this one deserves a bit more than just a "Yay!" — quasis are currently our way to represent code as ASTs, but it's like Perl 6's quasis bypass the standard way in Perl to quote code:

    AST form    ........^..................^........
                        |                  |
    lambda form ........|..................|........
                        |
    real code   ........|...........................
                p6's current way      v's codeToNode

Rakudo's quasi implementation has already validated how much quasis piggyback on blocks/lambdas. v's macro system validates that in a more direct way.

Con: There are now extra moving parts. We have to make up new variables $x and $y just so we can substitute them. The reader is expected to mentally do the substitution in .subst to see what the eventual code will be.

Pro: There's a neat sleight-of-hand that goes on there that you won't have thought about until I point it out: we feel like we're swapping out variables, but what we're really doing is substituting pieces of AST. It's that subtle difference between "the value in $x" (which does not even enter into the picture on this level), "the variable $x" (the thing we feel we're manipulating), and "the piece of AST denoting the variable $x" (the thing we are actually manipulating).

Con: We're finding $x and $y using short strings of text. Eww. It's partly a scoping thing. $x and $y are already out of scope, and so we cannot refer to them directly. It's partly a convenience thing. Strings are the simplest way to state what we are looking for. In fact, we don't really have a symbol type in Perl 6 that we could use instead. But it still feels wrong.

Pro: The code inside of the block is really undisturbed. Plus, I could imagine someone using this to her advantage, naming the variables in suggestive ways to either show their symmetry with the things we want to substitute, or show their function as placeholder variables. I would take the undisturbed swap statement any day over the hectic mess of triple braces.

Pro: It strikes me that we don't need to substitute just variables, we could substitute operators, too. Still need to figure out a way to accept them into the macro in the first place, though.

Con: Does this whole thing remind you of beta reduction in lambda calculus? Yeah, me too. The thing we run into then is capture-avoiding substitutions. To give a concrete example:

my $greeting = Q.valToNode("OH HAI");
Q.codeToNode({
    say $s;
    sub foo($s) {
        say $s;
    }
    foo("two!");
}.subst(
    '$s' => $greeting,
);

When this AST is injected and executed, it should print "OH HAI" and then "two!" (because $s got substituted once), not print "OH HAI" and then fail to dispatch (becase $s got substituted three times).

So the thing about capture-avoiding substitutions can be summarized as "shallow occurrences are fine, but hands off if nested blocks bind something". This is not so much a show-stopper as a "must think about this" and "must write tests that melt brain".

Con: The thing that truly bothers me, though, is this:

macro example($in1, $in2) {
    Q.codeToNode({
        say $x;
        say $y;
    }).subst(
        '$x' => $in1,
        '$y' => $in2,
    );
}

my $x = 5;
my $y = 42;
example($y + $y, $x);

I guess I would expect this code to spit out first 84 and then 5. But I fear it might well embark on a holy journey of infinite substitution, or maybe compromise somehow and output 10 and 5 or something. Someone is welcome to explain to me how my worries are unfounded. But from what I can see, this macro system suffers from the kind of naming clashes that gensymming seeks to avoid.

So... that's where I'll leave things today. I still think that this fairly simple macro system is interesting in what it teaches us about our own.

By the way, it's quite amazing how close it is to Perl 6's own macro system. I've studied a fair number of them at this point, and this one is very close. Which is kinda weird, with Perl 6 being what it is mixing compile and runtime, and CoffeeScript being what it is, compiling down to JavaScript and then getting out of the way.

Maybe we will ditch quasi blocks. Maybe we'll keep them. Either way, it's clear that we will get rid of the {{{ }}} syntax in favor of something else. This blog post shows the contours of that other thing, but the contours still remain fuzzy.

Not addressed by this proposal

Identified in a previous post.

Addressed (yay!) by this proposal

Macros: "Define Your Own Literals" Day

There's clearly a blurry line between macros and slangs. We should probably keep it that way. But for this post — as I keep mapping out more of the possible futures the best of which we are aiming for — I don't exactly know if it's macros I'm talking about, or slangs.

(Meanwhile, in a parallel universe, masak-with-a-toggled-goatee writes a similarly ambiguous post in his series of investigative blogginations about slangs.)

Consider this code from the near future:

use Slang::HTML;
use Slang::SQL;

my $keyword = "dugong";
my $db = $*DB;

my $webpage = html`
    <html>
        <body>
            <h1>Results for {$keyword}</h1>

            <ul id="results">
                {list-items($db.query( sql`
                    SELECT title, snippet
                        FROM products
                        WHERE {$keyword} in title
                `))}
            </ul>
        </body>
    </html>
`;

Let's ignore the fact that mixing HTML and database calls is something that most of us did back in the nineties and would prefer not doing again. Let's also leave aside the fact that I don't know the actual syntax for introducing a slang, and I basically just picked something that didn't look awful. (Or maybe it does. Anyway, it's not my call.)

Instead, let's focus on the three awesome things with the above code:

Where's all this coming from? I've had the above long-term vision for a long time, but it was re-triggered by the paper Safely Composable Type-Specific Languages. It's nice, although the first half is lighter reading than the second. My code above is based on their code from page 2, except mine has better indentation.

I do like the idea from that paper that literals could basically be defined by module authors. And there's another sense that stirs in me that I've had lately when writing hobby Perl 6: that of defining first a model (objects), then a set of rules (functions and methods), and then a syntax (custom operators). I've only had a small taste of that kind of programming, but I like it.

Anyway, here are my takeaways from the paper:

Possible ways forward on this: both the HTML and SQL slangs are possible today, with some degree of bending the implementation. FROGGS++ is showing the way with Slang::Tuxic — let's do something similar with these.

Edit: not one minute after publishing this post, I'm made aware of Slang::SQL on Github. It's not quite there yet, but it's working its way toward the ideal I have in mind. Nice! Ok, so now I only have to rally the people to building Slang::HTML. 哈哈

Signing off with this thought:

Any sufficiently complicated syntax highlighter for Perl 6 contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of STD.pm6.