Why LSD?
Yev Barkalov
October 26, 2024
In the 90s, the first commercially available web browser unlocked the capability for humans to use the internet. However, we somehow ended up in a world in which people are instead used by the internet. On LSD, individuals and organizations alike can finally stop tripping over web data and use the internet again.
What is LSD?
LSD is either short for “Live Structured Data” or “DSL” written backwards, feel free to choose whichever you prefer or make up a new zinger. What we provide is a holistic development environment that, with a postgres-compatible database and Chromium-based browser, reduces the tediousness of transforming data on websites into formats you can seamlessly insert into your workflows.
Every couple years a team comes along and attempts to tackle the semantic web problem however we believe prior attempts ultimately fail due to the problem of competing standards; there simply cannot exist a one-size fits all ontology for representing how people would like to interpret content on the web. Where we differentiate ourselves is by leveraging SQL-as-UI since the grammar of a SELECT statement already implies the desired output structure without the need for creating a table beforehand.
Like how Ruby on Rails is technically a web framework but feels more like a DSL for web development, you can look at LSD as technically being a postgres-compatible data source but feels more like a DSL for scraping the web. The difference between how easy it is to navigate a browser to a page and point your cursor at a field of interest is profoundly distinct from the effort it takes to spin up/deploy/extract with your own defined scraping script.
Who is LSD for?
Timothy Leary would say everyone and we’d agree but not for the reason he or you would expect. The world’s been undeniably changed thanks to the release of ChatGPT and the access to expert knowledge that’s now more attainable at people’s fingertips and we recognize the same holds true for being able to code. Ten years ago there was a clear line between being someone who’s technical and being someone who’s nontechnical but now even eight year olds can build and ship an end to end web application without breaking a sweat or yelling at a compiler.
If you’re a developer who’s found defining Selenium or Puppeteer scripts to be tedious; or maybe you’re a developer who’s gotten accustomed to it but finds it amusing that we keep needing to make new automated browser testing frameworks or protocols for operating browsers because of the lack of cohesion in the abstraction layer being accomplished, then LSD is for you.
Or, perhaps you’re a leader at your company responsible for extracting data off the web but do not see your differentiation being in the means of obtaining the said data. LSD can help your organization get results from internet extraction workflows in seconds instead of days because of the inherent waterfall development philosophy that tends to arise in internal scraping processes.
What does LSD believe?
The past several years saw the rise of the argument surrounding a “right to repair” and whether or not consumers should be able to access or modify the internals of products they buy and use. From a bird’s eye view it’s really just a continuation of Stallman’s arguments against JavaScript and the unknowability of what your applications are really doing on your computer. Instead, recognizing the tech-forward world we live in as well as what Cambridge Analytica reveals about people’s actual preferences, we believe in a world that embraces a “right to code” where technology that represents or extends your capabilities should be easily programmable.
Like how somebody who is semi-professional at cooking can view a dish at a restaurant and know how to recreate that meal, there is an unspoken superpower of people who know how JavaScript works in websites and how seamless it can be to pluck out data of interest. This is not a capacity that should forever remain in the hands of arrogant software engineers who succumb to golden handcuffs.
Over a decade ago we were promised that, in five years, we wouldn’t need a driver’s license since autonomous driving will be solved by then and, here we are still waiting for robo taxis to be a widely available safe service. Now everyone is wanting for autonomous agents to exist in the loop in order to make their lives easier, how can we sculpt a world in which the means to leverage autonomous actions are readily available?
What is the world LSD is building?
We’re working towards a future in which actions people take on the web are both composable (so people can leverage what’s already been defined, effectively standing on the shoulders of giants) as well as decomposable (so people can deconstruct/modify without looking at a workflow like a black box). For a condensed representation of how we’ve already tackled one of the hard parts of this problem, our recent demo of operating a browser with a caching strategy that is aware of page altering operations showcases an underlying homotopy type theory of web browsing if you consider the following definition for the web: a group containing all pages in specific states (ie google.com, google.com following entering a query, google.com following entering a query and pressing the search button).
Each path consisting of page altering operations can be viewed as being similar to sequences of moves applied to a Rubick’s cube which offers rich capabilities in our SQL. If any of the above interest you, follow us @getlsd for company updates or @itisyev for product updates!