Integrating advanced filtering in a web application: Common Expression Language Part 2

For this article, I asked DALL-E Mini to make a picture of Node.js

In my previous article, I discussed how I built the Common Expression Language spec for TypeScript so that it could be run in a browser for complex queries.

Now it’s time for part 2, where I discuss how I actually integrated this into my web app.

In terms of the deployment, there are only a few exports in the index.ts file.

I published the project onto NPM to make it easy to install in my various projects.

npm install --save @fleker/cel-js

Web Workers

Since I’m running a bunch of operations, I was interested in adopting web workers, an easy way to run a task in a non-blocking thread.

However, after spending about an hour struggling, I decided against it. The rules around web workers are a bit complicated in a modern development environment. It wasn’t clear how I’d be able to import a third-party library. Then I need a way to have it work with an Angular component.

I went down a rabbit hole for a bit until I shrugged. It seems like I could install some additional build tools solely for this web worker, but then that’s an entirely different process from the Angular build tools. It would’ve been a headache to manage everything.

So this evaluation will run in the main thread. If that blocks page execution, so be it.

Angular Service

Since I may be using CEL evaluations in multiple components, I decided it would be best to put everything into a service as I’m using Angular. This aptly-named CelService has just a single method, run , which takes in an array of entries and returns only the ones that evaluate to true in the expression.

I can add as many attributes to the bindings as I want based on my entry interface. Note that I do need to be very explicit with the types. Anything that is a string needs to be wrapped in a string before being sent to the TextFormatter. Arrays also need to be formatted specifically.

For every entry, I run the format method on the user’s input expr with the parsed bindings. These bindings need to be in the same format shown in the previous article.

Once I get back these filtered entries, I can present those back to the user in the component.

Angular Component

So when my list component loads, every entry will be shown. The user can then type in a query that they want and hit the Run button. Then the expression will be run on every single entry and only the valid entries will be shown in the list.

This code snippet above is the end result which I implemented after a bit of trial and error.

First you see I’m setting some exec.runCel value to be true. This ties into the design. As this process ends up taking over a second, I want to make sure the user sees some acknowledgement. This variable, when true, will disable the button. Once the evaluation is done, the button returns to being clickable.

It is also for this reason that I am running this in a timeout after fifteen milliseconds. That’s just enough time for the button state to change before evaluating. If there was no delay, the button would never appear to change and the page would freeze for no apparent reason.

I also am saving the user’s last query in the browser’s local storage. As these queries can be long, saving and recalling them in a future session saves time having to write these out each time.

You can see in my HTML template that once the filteredEntries gets updated, the list of items on the page will change.

Performance

Performance-wise, it’s not bad. Based on my tests, on an admittedly good desktop, I can filter several thousand objects in about a second. Unfortunately being on the main thread, it means the page freezes for that second.

This filtering complexity is O(N), for N entries. But then it’s also O(M) to compute the bindings for the fields of each entry and even more time to evaluate the whole thing. Overall, it’s O(N²), much more complex than just comparing against plain strings.

I turned it into a setting that the user can enable or disable. Some people may prefer the additional complexity, while others may prefer performance. Ideally you can get the best of both, but right now I don’t have a good way to determine which path to take.

Wrapping up

Not only did we build a high-quality, robust expressions library, we have incorporated it into the framework of a larger web app.

There are definitely opportunities for improvement. I can imagine several different performance improvements that would make each query less costly. But it does fulfill one goal of being able to run in the browser rather than relying on a server. Server calls can take more time and won’t work offline. At least here we can be sure the user gets something back.

There’s also plenty of opportunity to improve the underlying grammar. As stated in the previous article, CEL is a fairly large spec that can do much more than the little I’ve implemented. Expanding this feature set might come at the cost of performance, but these are always going to be competing trade-offs.

If you are interested in picking up this tool for yourself, check out the repo and install it from NPM.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store