Voucher collection fetch takes a lot of time with the new beta release(1.0.40)

[anksgupta]

When i use the json based approach, the voucher collection fetch takes a lot of time. Attaching a screenshot for your reference.

Is this expected or are there any more performance improvements that are yet to be published?

Btw below is the config.json object:

{
"database": {
"technology": "csv",
"server": "localhost",
"port": 0,
"ssl": false,
"schema": "tallydb",
"username": "sa",
"password": "admin",
"loadmethod": "file"
},
"tally": {
"definition": "tally-export-config.json",
"server": "localhost",
"port": 9000,
"fromdate": "2025-04-01",
"todate": "2026-03-31",
"sync": "full",
"frequency": 0,
"company": ""
}
}

Thanks in Advance

[dhananjay1405]

@anksgupta yes indeed this is a bug, as I was notified by few more people who are facing similar issue. I am almost finished with fixing this bug on local DEV environment. Expect a build in next 2-3 days where this issue would be fixed.

[dhananjay1405]

@anksgupta this issue is now resolved in release 1.0.41, request to grab (download) the latest release. Let me know if the issue still persists

[anksgupta]

@dhananjay1405 I downloaded the latest code. Below are few observations w.r.t a company loaded in tally with 12039 vouchers:

1. with batchsize: 6000, none of the batches return any response.
2. with batchsize: 5000, the first batch downloads successfully and the rest of the 2 batches don't return any response.
3. with batchsize: 4000, none of the batches return any response.
4. with batchsize: 3000, none of the batches return any response.
5. with batchsize: 2000, none of the batches return any response.

Also if i reduce the period in config.json, I am seeing inconsistent results. To give you some more context, I modified tally.toDate to "2026-10-31". So after reducing the period and setting batchSize to either 2000 or 3000, the first 2 batches don't return any results and the 3rd batch returns all the results.

[dhananjay1405]

@anksgupta , let me share some information based on my testing, which might help you troubleshoot issue

• Setting batchsize more than 10000 mostly ended-up process getting stuck, since tally.exe never responded back (as their internal buffer gets exhausted).
• Optimal batch size was found to be 5000, but again I am not aware about the system configuration you are using, since I tested this approach on just my personal system
• Quoting an example, if utility splits it into 5 batches, response returned in batches is cached in memory (RAM), and after completion of all the batches, rows returned in individual batch are combined. So, you might feel that only last batch is returning back response
• I tested this on demo company with around 45000 vouchers, where batchsize 5000 worked perfectly
• Development and testing were done on my personal PC with configuration of 32 GB RAM, out of which, tally.exe was using only max 4 GB RAM throughout the export. As per my estimate, if your PC is having configuration of 8 GB RAM or less, then this approach is bound to get stuck, and you might not see any results
• Ensure to run this on paid (Silver / Gold) version of Tally Prime. Educational version might not support this, since internally utility is dynamically adjusting from/to date in the back-end code, to limit individual batch to max batchsize (since tally.exe is running into issue of buffer exhaustion which results in process getting stuck indefinitely)

[VikasGupta1806]

Experienced a similar issue with the version 1.0.40. When I configured the utility to use the json method, the utility exported xml instead of CSV, that too at a very slow speed.

Now downloaded the 1.0.41 version. Tried exporting for a small date range of 1.5 month. The process is stuck at initial step. Here is the config file snippet:
{
"database": {
"technology": "csv",
"server": "localhost",
"port": 0,
"ssl": false,
"schema": "tallydb",
"username": "sa",
"password": "admin",
"loadmethod": "file"
},
"tally": {
"definition": "tally-export-config.yaml",
"server": "localhost",
"port": 9000,
"fromdate": "2026-01-01",
"todate": "2026-02-19",
"sync": "full",
"batchsize": 1000,
"frequency": 0,
"company": ""
}
}
I am running this on a server with ram in the range of 32 gb-64 gb or more. for further context, I have a read-only right in the tally server.
Thanks in advance!!

[anksgupta]

@anksgupta , let me share some information based on my testing, which might help you troubleshoot issue

• Setting batchsize more than 10000 mostly ended-up process getting stuck, since tally.exe never responded back (as their internal buffer gets exhausted).
• Optimal batch size was found to be 5000, but again I am not aware about the system configuration you are using, since I tested this approach on just my personal system
• Quoting an example, if utility splits it into 5 batches, response returned in batches is cached in memory (RAM), and after completion of all the batches, rows returned in individual batch are combined. So, you might feel that only last batch is returning back response
• I tested this on demo company with around 45000 vouchers, where batchsize 5000 worked perfectly
• Development and testing were done on my personal PC with configuration of 32 GB RAM, out of which, tally.exe was using only max 4 GB RAM throughout the export. As per my estimate, if your PC is having configuration of 8 GB RAM or less, then this approach is bound to get stuck, and you might not see any results
• Ensure to run this on paid (Silver / Gold) version of Tally Prime. Educational version might not support this, since internally utility is dynamically adjusting from/to date in the back-end code, to limit individual batch to max batchsize (since tally.exe is running into issue of buffer exhaustion which results in process getting stuck indefinitely)

@dhananjay1405 The API calls are made sequentially in nodejs so the batches are also loaded sequentially. I've added log statements in tally.mjs to show you how the first batch returns records but the next 2 don't.
Attaching a screenshot for your reference.

[dhananjay1405]

Experienced a similar issue with the version 1.0.40. When I configured the utility to use the json method, the utility exported xml instead of CSV, that too at a very slow speed.

Now downloaded the 1.0.41 version. Tried exporting for a small date range of 1.5 month. The process is stuck at initial step. Here is the config file snippet: { "database": { "technology": "csv", "server": "localhost", "port": 0, "ssl": false, "schema": "tallydb", "username": "sa", "password": "admin", "loadmethod": "file" }, "tally": { "definition": "tally-export-config.yaml", "server": "localhost", "port": 9000, "fromdate": "2026-01-01", "todate": "2026-02-19", "sync": "full", "batchsize": 1000, "frequency": 0, "company": "" } } I am running this on a server with ram in the range of 32 gb-64 gb or more. for further context, I have a read-only right in the tally server. Thanks in advance!!

@VikasGupta1806 I can see that in your config.json you are using yaml definintion, which is using the same old mechanism of extraction. The only changes I applied in the old method is setting from / to date by issuing command to tally which mimics Alt + F2 change period option of tally. YAML option is not using the batchsize approach and instead pulling entire from/to date. JSON method pulls data in batches by dynamically adjusting from / to date in the backend code. So, if you are using YAML, technically utility should behave just like the older version. But since you are facing issue, I will again look at code to see, if there is any impacting thing on YAML stuff during implementation of that collection-based extraction (JSON method).

[dhananjay1405]

@anksgupta going by the screenshot, I think now I have a better understanding of the issue. The XML request sent to Tally in batches seems to be successful. But from second batch onward, there seems to be parsing error. Basically, the code attempts to extract individual voucher collections from the XML and parse into JSON object, which are added to array. Once all the batch are completed, based on this array, either CSV or database INSERT statements are prepared. I can clearly see parsing error.

Let us try to troubleshoot and find out if any invalid or unreadable character (returned by Tally) is causing JSON to break. Open tally.mjs find the first line of code (Line 478) and insert one more line (2nd line) as shared below

lstVoucherCollectionData.push(...await this.parseXmlToJsonCollection(targetCollection.collection));
fs.writeFileSync(`./csv/voucher_${batchCtr}.json`, JSON.stringify(await this.parseXmlToJsonCollection(targetCollection.collection))); //save batch file
fs.unlinkSync(`./csv/voucher.xml`); //delete temporary file

In short, we are logging the JSON object batch by batch. Use JSON validate (available on online websites) to check if an object array of voucher collections is valid or it breaks due to unreadable character

Let me know the result

[anksgupta]

@anksgupta going by the screenshot, I think now I have a better understanding of the issue. The XML request sent to Tally in batches seems to be successful. But from second batch onward, there seems to be parsing error. Basically, the code attempts to extract individual voucher collections from the XML and parse into JSON object, which are added to array. Once all the batch are completed, based on this array, either CSV or database INSERT statements are prepared. I can clearly see parsing error.

Let us try to troubleshoot and find out if any invalid or unreadable character (returned by Tally) is causing JSON to break. Open tally.mjs find the first line of code (Line 478) and insert one more line (2nd line) as shared below

lstVoucherCollectionData.push(...await this.parseXmlToJsonCollection(targetCollection.collection));
fs.writeFileSync(./csv/voucher_${batchCtr}.json, JSON.stringify(await this.parseXmlToJsonCollection(targetCollection.collection))); //save batch file
fs.unlinkSync(./csv/voucher.xml); //delete temporary file
In short, we are logging the JSON object batch by batch. Use JSON validate (available on online websites) to check if an object array of voucher collections is valid or it breaks due to unreadable character

Let me know the result

@dhananjay1405 I've already saved & checked the XML response of each batch. I can see the list of vouchers in the first batch's XML file but the second & third batch's XML files don't have any vouchers. There is no parsing error, the XML response itself is empty which means records are not returned from tally.

[dhananjay1405]

@anksgupta if you are getting empty response from tally, then indeed this is another interesting bug. For me, setting batchsize beyond certain number caused random batch to hang-up tally indefinitely without returning back any response. I remember one more user getting similar bug with error invalid number in content-length error.

I need to re-produce this bug on my development PC, which can help me find precise issue. I managed to extract 40k vouchers in one of the tally companies by playing with batchsize number between 5000 to 10000. Let me think how we can identify exact cause. I will request you to install a tool called Postman in your PC. I will try to share you few postman HTTP Request payload, which you can fire on your Tally to experiment and inspect where the things break. Request to reach me out over my email info@excelkida.com with Hi, so that I can email you back some Postman payloads. I will avoid sharing it here, for the sake of data privacy.

As of now, the only thing I can suggest is to use YAML as a fallback, which has been tested over time.

[anksgupta]

@dhananjay1405 thanks for looking into this. For now I will go with the YAML approach. Do you have any plans for implementing JSON based extraction instead of XML based extraction?

[dhananjay1405]

@anksgupta I did try JSON based extraction when Tally Prime Release 7.0 was released around 3-4 months ago. But the JSON extraction is slower than XML, which is strange. I have even emailed this observation to Tally Solutions development team. Unfortunately, I had to revert back my entire JSON based extraction code due to this, as 2x speed matters a lot (I never pushed that code, it is still lying in a hidden directory on my local copy of this repo which I can anytime plug back into codebase). JSON based extraction is best suited if you intend to develop direct integration of any CRM with ERP, since majority of libraries support JSON request / response.

[anksgupta]

@dhananjay1405 Got it. Thank you.