Improving CEL performance with streaming API
Last summer I talked about a library I built to run Common Expression Language operations in the browser. This is a way to write simple or complex queries against a dataset.
I mentioned at the time some performance issues I had faced, since the operation can take several seconds and block the main UI thread during that time. This means that nothing on screen will register user inputs until the query finishes. That’s not a great user experience.
Since then I’ve added a number of improvements when using this library that I want to share. You can consider this to be a Part 3: using second-order design to make the query experience a lot better.
First, let’s consider a Pokémon game. In this game, players can catch a variety of Pokémon that get placed in their collection. In reality, this collection is represented as a map. The key is the ID of the Pokémon: an encoded string that contains all their properties. The value is a number, how many of that Pokémon ID they have caught.
Players will have a variety of reasons to run queries against their collection. They may search by name, their types, their known moves, their gender, their form, whether they are shiny, or a variety of other attributes. Anyone who has played the game knows the variety of ways you can search.
This is a great reason to adopt Common Expression Language in the game: rather than trying to create custom handlers for each of these, I can wrap all of a Pokémon’s attributes into a bindings object then pass that to the CEL text formatter. If it returns true, then that Pokémon is one of the entries returned in the final array.
The code to do all of this is above. One downside to consider here is that this needs to execute for every single key in the player’s collection. That means it will run potentially thousands of times. It also requires a variety of memory-intensive procedures like constructing a Badge object to decode the map key and looking up species attributes with the get function.
Some of this is unavoidable, though some attributes like family can be pre-generated as well. Nevertheless, players may be waiting 10 seconds or longer for this to complete. Since it blocks all UI interactions, they have no choice but to wait.
Over the last year, I’ve put together three additional steps to make queries better.
Yielding
Rather than rushing through thousands of filter operations all at once, it might make sense to do a bundle then take a brief pause and allow for UI events to proceed. This is a suggestion from the Chrome Dev Blog.
Each filter operation wraps a function into an array of tasks. Then each task runs. However, every 100th task triggers a pause for other UI events. This leads to improvements in general responsiveness.
However, players may stil have to wait 10 seconds or longer for queries to complete. If they made a typo or want a different query, they need to wait for it to complete before starting a new one.
Quick Aborts
This can be mitigated by introducing a second field to the Task object: a runId. This identifier is simply tied to a timestamp when the operation begins.
Now when each task is popped off the queue, it is verified to be associated with the current runId before executing. If it isn't, the task is quietly discarded which saves a lot of time.
A button can now be placed in the UI which calls the stop method: this modifies the runId so all existing tasks are immediately ignored. Thanks to the yielding support above, players are actually able to interact with the UI while a query is happening.
When stop is called, the run operation returns instantly with all of the filteredEntries it managed to get to by that point. But why do I have to wait to get a partial result? Is there a way that I can start getting results before it's done?
Streaming Results
Another change modifies the way that results are returned. Up to this point, this was a long-running asynchronous function.
Here, I just call the run method with all the entries and the user-input filter. Then the result would be returned as data and update the UI at the end.
This is later updated to a new runAndSubcribe method. Rather than return a list of results at the end of executing, I return a Subject from rxjs. Every successful match immediately sends a new event back to the picker widget.
Now, results start to stream in immediately after running. This means players feel a greater responsiveness.
Additionally, I now pass a percent complete field that I can use to update a progress bar on the frontend. This improves that responsiveness.
You can see the changes in the picker behavior too. The subcriber is set so that picker fields get modified on each event. These fields are then reflected in the final UI.
Conclusion
This CEL library has a lot of capabilities all on its own. In fact I didn’t need to change anything to get any of these improvements in the quality of life.
This is a good example of why frontend design matters. You can have a great backend that is rich in features, but that matters little if your clients do not enjoy using it. Considering the user experience is something of perennial importance. If you can find ways to make their lives better, it’ll continue to pay dividends.
