Is there a way to run report setup in background?

janet commented on May 31, 2021

Hello! I have a huge report, for which i need several data sources, some of them are API's, and they work rather slow. I wish it was any possibility to run separate reports for every API in background by cron, for example, saving data in cache or local DB, so that the complex report could get data faster. Is there any workaround?

I guess, maybe separating data sources to different reports and getting data from them, as described here - https://www.koolreport.com/data-sources#reportdatasource could help a little, as all API's will work kinda synchroniosly, but if there're any better decision?

Sebastian Morales commented on Jun 1, 2021

Janet, you would need a persistent storage to save your background processed data. The storage could be temporary files or temporary databases. And separate your reports to the background running ones and the real time displaying ones.

A work flow might be like this:

Cron: BackgroundReport_1->run() => save BackgroundReport_1's datastores to the persistent storage.
Cron: BackgroundReport_2->run() => save BackgroundReport_2's datastores to the persistent storage.
...

UI: RealtimeReport_1 reads data from the persistent storage => displays result.
UI: RealtimeReport_2 reads data from the persistent storage => displays result.
...

Let us know if you have further questions. Rgds,

janet commented on Jun 1, 2021

Sebastian, thank you, it's what i was thinking about. The question is, - how can i save report results to database, for example, what is the best practice? Then i could just add that db as datasource for my Big report.

I guess in ->run()->render() i should use something else but render?

Sebastian Morales commented on Jun 1, 2021

Janet, a report's result basically includes its datastores after the run() method is called. A datastore consists of meta and data using which you could build the datastore again. Some sample code to illustrate the idea:

    include "path/to/MyReport.php";
    $report = new MyReport();
    $report->run(); //no need to render here as we only care about saving the report's data, not viewing it in widgets

    $datastore1 = $report->dataStore("datastore1");
    $meta1 = $datastore1->meta();
    $data1 = $datastore1->data();

The meta and data variables are array (print them out if you want to have a look) which you could then save to files or databases in table or JSON form.

One important notice: if these background reports/tasks take a long time to run you don't want to run it from beginning of data sources every run. For each run, save the report's data together with the run timestamp. The next time the report runs you want it to begin with data starting from the previous timestamp.

janet commented on Jun 1, 2021

KK, i see. And what about saving it in databases, shall i use raw php connect to DB and push it there, or there's any functionality as in reports. In other words, can i connect to db the same pretty way i connect to it as to dataSource, but in this case in shall be dataReciever?

Sebastian Morales commented on Jun 2, 2021

Pls create your own temporary database and write data read/write function for it. We would consider adding write function support for our datasources in the future. Tks,

janet commented on Jun 2, 2021

Thank you! It would be cool! 1. To make dataReceiver 2. To make ability for piping data through another dataSource, so that it comes though cron to temporary one, and goes to report from the temporary

One more question, - is in possible to render report right to csv?

Sebastian Morales commented on Jun 2, 2021

While it's not rendering to csv, you could use export to csv and either show the csv to browser:

https://www.koolreport.com/docs/excel/export_to_csv/

or save it to your server by changing ->toBrowser() to ->saveAs($filePathOnServer)

KoolReport's Forum

Is there a way to run report setup in background? #2114

Build Your Excellent Data Report