The pipe operator ‘|>’ syntax is a way to write SQL that is more readable and easier to understand.
Inspired by a paper recently published by Google where they go over adding a pipe operator syntax to their database language, LSD SQL features a pipe operator syntax to improve the readability of programs that define wrangling the web.
FROM starting_point
|> A KEYWORD with arguments
|> ANOTHER KEYWORD with more arguments...
If you’ve dealt with convoluted CTEs or tried to nest multiple EXISTS clauses, then you’ll know how complex SQL can sometimes look similar to a deeply stacked LISP program hence the language design decision here. The pipe operator syntax enables developers to write functional SQL to transform web data in a way that feels less like hand-holding an automated browser and more like remixing data to reach your desired structure.
We also use the pipe operator to assign variables.
To translate from standard SQL, break up the clauses into expressions and separate them with a pipe operator while starting with a FROM. For example, let’s take the following query in standard syntax:
FROM
https://news.ycombinator.com
SELECT
a AS post
, a@href AS link
GROUP BY
span.titleline;
These are the clauses that make up the statement:
[FROM
https://news.ycombinator.com]
[SELECT
a AS post
, a@href AS link]
[GROUP BY
span.titleline;]
So in pipe operator syntax that could look like:
FROM https://news.ycombinator.com
|> SELECT a AS post, a@href AS link
|> GROUP BY span.titleline
The order of expressions can be interchangeable so long as they don’t alter the page you’re interested in meaning the below statement returns the same data as the one above.
FROM https://news.ycombinator.com
|> GROUP BY span.titleline
|> SELECT a AS post, a@href AS link