top of page

Limited Concurrency Api Aggregation

Updated: Jul 7, 2023


Most front-end applications use REST APIs for backend communication and depending upon the architecture (micro-services or not) the number of apis involved in the communication as a whole can vary a lot.


In this article we’ll be talking about one of the most basic scenarios that majority of the applications have in abundance – listing of data.


Listing apis generally provide pagination to optimise consuming small chunks of data and if the front end needs also fit this then nothing better. But nevertheless, in some scenarios there might be a need to get hold of data across a bunch of pages and getting all in one go might not be favourable because of a variety of reasons including but not limited to execution timeouts, limited network bandwidth etc.


 

Note on concurrency:

Javascript does not have the notion of concurrent threads as the execution of the event loop is single threaded. For the context of this discussion, concurrent means pending/inflight operations at a given point in time. This definition of concurrency works here since the actual work involved is limited to initiating api calls and then waiting for the response before moving ahead. The term worker refers to a sequence of operations that defines a logical thread.


 

Here are a couple of simple options to fetch multiple pages with the drawbacks of each:


  • Sequential Read: This option is the simplest and the slowest of the lot but suffers from increasing delay with growing number of pages as all the pages will be retrieved one after the other.

  • All At Once: This one can be the fastest as it will fire read operations for all the pages at once but can be least efficient and most error prone if backend resources and network bandwidth are on a crunch.


To tackle this what we need is a way to express the permutations across api call concurrency, max chunk in a single call etc. to make sure the consumption happens in a controlled way and can be tweaked as per needs.



Implementation breakdown

The implementation consists of a central orchestrator that drives the behaviour depending upon the parameters provided(options to tweak performance and load) and has dependencies of custom implementations for IRemoteService(api interactions) and IOrchestrationMeta(consumer side updates). All of these pieces are described below in detail.


IRemoteService:

This is the abstraction over a remote API that is to provide data for consumption and has to be custom implemented to communicate with a backend api.


The request and response models for the fetchAsync operation in this have been defined with some basic parameters that will be read by the library for managing state and can be extended for any custom requirements that come in the implementation of IRemoteService.


IOrchestrationMeta:

This is a consumer side hook to get some information at different stages of the operation and for now just notifies of the actual concurrency happening for an operation.


RemoteAggregator:

This is the aggregator implementation that takes care of the orchestration around concurrency and load.


It has a dependency on an instance of IRemoteService.


Following are the parameters that determine the degree of concurrency and load factor:

  • maxWorkers: This is the max level of concurrency allowed. Depending upon the amount of data to be retrieved, this can range from 1 to a max of the value passed in this parameter.

  • unitPageSize: This is the max number of items to retrieve in any of the calls being made to retrieve data.


Here’s a visual representation of the entire thing together:





 
The source code can be found here.

It also includes a couple of test cases to demonstrate the functionality.


The source code is in typescript and can be used across NodeJS and browser environments alike.


147 views6 comments

Recent Posts

See All

6 Comments


Manak
Manak
Mar 15, 2023

Thanks for sharing

Like

Shivam Dubey
Shivam Dubey
Jan 28, 2023

The content is very clean and understood.

Like

Abhilekh Choudhary
Abhilekh Choudhary
Jan 28, 2023

Nice

Like

kumar waghmode
kumar waghmode
Jan 27, 2023

Nice attempt to move backend distribution logic to frontend to reduce backend resource utilisation. Do you have any real time example where this library would be useful.


In a case where frontend needs to fetch many pages at once, websocket can be used to stream page data provided backend expose websocket endpoint.

Like
Rajesh Sharma
Rajesh Sharma
Jan 29, 2023
Replying to

This particular way of handling data consumption is more suitable in places wherein there is an existing REST API and there is a need to handle multi-page consumption(not something which is needed everywhere) without incurring change to the backend protocol. Web-sockets are good in terms of bi-directional communication but don't really play well with serverless architectures due to the persistent nature of the connection.

This article tries to give an alternative(one of many) to a use case that is neither as regular as the needs of a listing page nor as extreme as expecting a complete export of all the data. Its good to have options available in terms of approaches so the right one can be put in place…

Like

AAKARSHIT GIRI
AAKARSHIT GIRI
Jan 27, 2023

Great initiative

Like
bottom of page