Abnormal termination occurred while processing large files

[he113-code]

The version is 5.8.0
My 3D image sizes are (1751, 3554, 2524)
It is a binary 0-1 mask

run code:

skels = kimimaro.skeletonize(
labeled_array,
parallel=1, # <= 0 all cpu, 1 single process, 2+ multiprocess

)
print(skels)

console output ：
The process has ended, the exit code is -1073741819 (0xC0000005)
Can I get everyone's help @william-silversmith

[he113-code]

I trimmed a small portion of data (296, 1411, 2007), and the cropped data runs well. My memory is very sufficient, with 1.5T of memory.

[he113-code]

I found through positioning that the function failed to execute, while the previous ones performed well

D:\Anaconda3\envs\py311\Lib\site-packages\kimimaro\intake.py

def edtfn(labels):
    if isinstance(labels, CrackleArray):
      labels = labels[:]

    return edt.edt(labels, 
      anisotropy=anisotropy,
      black_border=(minlabel == maxlabel),
      parallel=parallel,
      voxel_graph=voxel_graph,
    )

  all_dbf = edtfn(cc_labels)

Unable to obtain all_debf result

[william-silversmith]

Hi! Thanks for finding this issue. What is the smallest size array that reproduces the error? If you're able to share the data, can you send it to me compressed as a crackle array? My development machine is a MacOS arm64 chip with 36 GB RAM.

https://github.com/seung-lab/crackle

[he113-code]

I have sent you the modified virtual data via emailwilliam.silversmith@gmail.com

[william-silversmith]

Thank you, I tried running it on my laptop. It got to the "Skeletonizing Labels" stage before getting OOM killed, which means it finished the distance transform part. That means that the issue you are seeing is likely architecture/compiler dependent (i.e. there is some ambiguity in the type system in the edt C++ code that gets interpreted differently by MSVC).

Are you able to do edt.edt(data) of the data directly? Does it error there?

[he113-code]

I am unable to obtain the result of edt. edt (data).I am running on Windows 10 Intel(R) Xeon(R) Gold 6246 CPU ``` import edt import numpy as np import tifffile data = tifffile.imread(r'F:\data.tif') out1 = edt.edt(data) print(np.unique(out1)) ``` The process has ended, the exit code is -1073741819 (0xC0000005)

[william-silversmith]

Would you mind getting the latest master branch of https://github.com/seung-lab/euclidean-distance-transform-3d and seeing if that works? I tried making the types more uniform and increased the precision of the calculation. It might even be 10% faster.

[he113-code]

image_slicing.py
Unfortunately, I am unable to complete the latest version of edt compilation and installation in Windows 10.
But I came up with the method of slicing the graph to obtain the results.
My method is based on prior knowledge that the diameter of my blood vessels/nerves will not exceed 20 pixels. So I believe that when the overlapping area of the top, bottom, left, right, left, and right of my cropped image is greater than 20, taking the middle processing result after cropping will be consistent with the original one.
I tried a pruning strategy based on xy and no pruning strategy in the z direction, and compared it with np. allclose (a, b). The results of the two methods were consistent.

When I expanded to three dimensions, that is, xyz adopted a cropping strategy at the same time, I found that I could not obtain consistent results, and I don't know where the problem lies. Do you think the code is feasible under the limited conditions known earlier for this strategy? I uploaded the code. Because the diameter of blood vessels/neurons is almost small, mostly 1-5 pixels. So when dealing with larger data, can we consider implementing this solution. A maximum diameter parameter can be separated and defined by the user, perhaps 10/50/100/200.

[william-silversmith]

That's so interesting that you found it was consistent. I am not sure why including z changed it. I have code in https://github.com/seung-lab/igneous/ for doing arbitrarily large volumes via chunked processing. Maybe that will help? However, it doesn't use extensive overlap, only 1 voxel and has a way to make ends meet exactly. I will try releasing new wheels for edt.

[he113-code]

I got the result when parallel=1, but when I set parallel=5, I found an error. I thought it was a Windows 10 environment issue, so I used Docker to set up a Linux environment. And installed Python 3.13, using pip install kimimaro for installation. According to the installation record, it shows: Downloading kimimaro-5.8.0-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (2.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.8/2.8 MB 5.2 MB/s eta 0:00:00

So it should have been pre compiled, but when I ran parallel=5 again, it still reported an error. I also tested connectomics.npy.ckl.gz in the example code
I only changed the parallel parameter, and testing found that when parallel=1, the execution time is 12-13 minutes

When parallel=5, the same error occurs.
Please refer to the screenshot for specific details.

[william-silversmith]

It looks like you are failing to allocate a shared memory array for some reason. That error is hard to read, but it's because dbf_mmap was never assigned in the first line of that function, an exception is raised, and then in the finally block, dbf_mmap.close() is called. I probably should check if dbf_mmap exists first so the proper exception will bubble up....

[william-silversmith]

Oh it says "no module named psutil" I missed that on the first read. try: pip install psutil

[he113-code]

I can't confirm the new error now

[william-silversmith]

This likely means that the shared memory was not allocated and an access to it triggered an OS error. This could be due to an adjustable setting that limits shared memory allocations on your machine.

…

On Wed, Nov 26, 2025 at 7:37 PM tt11 ***@***.***> wrote: *he113-code* left a comment (seung-lab/kimimaro#119) <#119 (comment)> 2.PNG (view on web) <https://github.com/user-attachments/assets/ea81bc8f-0ccb-46da-8cff-b3fd11735f3d> I can't confirm the new error now — Reply to this email directly, view it on GitHub <#119 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AATGQSNZNJKLZJPWX66ZU5L36ZBUZAVCNFSM6AAAAACNDP2F5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTKOBTGY2TCNZZGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[he113-code]

Is this error on the system? Can you fix it by updating the code? I don't know how to make shared memory assignable, can you reproduce it on your end? I am using sample data and sample code