feat: add .ts_ignore pattern ignoring system

[CyanVoxel]

Summary

Replaces the existing "File Extension List" inclusion/exclusion system with a more comprehensive ".gitignore" style pattern ignoring system. The built-in GLOBAL_IGNORE_SET has also been replaced with this same pattern-matching system, both for directory scanning and query filtering. This PR also adds the ability for TagStudio to traverse symlinks (not tested on Windows) and Unix hidden folders. I was originally planning on those changes to be in a separate PR, but most of the code overlapped with these changes and ended up amounting to a couple of flags passed to the wcmatch glob function.

How to Use

1. Create a .ts_ignore text file located directly inside the .TagStudio folder of one of your libraries (library can be open!)
2. Use .gitignore-style pattern formats to exclude files and folders, with each pattern being on a new line.
3. Save the file and refresh your TagStudio directories and/or search to test the patterns. The file can be edited live, with changes reflected in subsequent searches and scans.

Example .ts_ignore file:

# TagStudio .ts_ignore file.

# Code
__pycache__
.pytest_cache
.venv
.vs

# Projects
Minecraft/**/Metadata
Minecraft/Website
!Minecraft/Website/*.png
!Minecraft/Website/*.css

# Documents
*.doc
*.docx
*.ppt
*.pptx
*.xls
*.xlsx

TODO

• Remove extension list feature, generate .ts_ignore file from existing configurations
• Add frontend .ts_ignore editor
• Add documentation
• Add tests
• Test for character escaping

Tasks Completed

• Platforms Tested:
  • Windows x86
  • Windows ARM
  • macOS x86
  • macOS ARM
  • Linux x86
  • Linux ARM
• Tested For:
  • Basic functionality
  • PyInstaller executable

---

Closes #14, Closes #624.

[Pheubel]

I've tried out the PR in it's current state on Windows and did some quick testing.

generally speaking it seems to work, but when i add comments to the .ts_ignore file it causes the ignore file to break and the listed patterns all stop working. Although i am not sure if adding comments are needed, since the plan is to have this file be altered by the program itself. since the example made use of them, i thought it was worth mentioning it.

Another thing that should probably be addressed is: files that were previously discovered, but are now ignored by the ignore pattern. A this point, files that were previously included in the library will stay included even after they should be ignored. In my opinion, this is not a bad thing. However, it should be visible that the file is currently included but should be excluded. Perhaps by adding a warning icon that when hovered tells the user that the current file is being tracked. The user should then be able to decide to remove the file from the library, without removing it from the file system

Side note, perhaps it is worth breaking out some of the TODO's into their own issues/PR's so that this one can become more focused on the core feature.

[CyanVoxel]

generally speaking it seems to work, but when i add comments to the .ts_ignore file it causes the ignore file to break and the listed patterns all stop working.

Strange, I've been using comments like the ones in the example in my libraries on this branch with no issue. Is there something about your comments that may differ from the examples?

Although i am not sure if adding comments are needed, since the plan is to have this file be altered by the program itself. since the example made use of them, i thought it was worth mentioning it.

These files are intended to be both editable from within and outside of the program, so working comments would indeed be desired.

Another thing that should probably be addressed is: files that were previously discovered, but are now ignored by the ignore pattern. A this point, files that were previously included in the library will stay included even after they should be ignored. In my opinion, this is not a bad thing. However, it should be visible that the file is currently included but should be excluded. Perhaps by adding a warning icon that when hovered tells the user that the current file is being tracked. The user should then be able to decide to remove the file from the library, without removing it from the file system

By "at this point", are you referring to the most recent stable build (9.5.2) or this PR? Because in this PR the intended behavior is to not have files that should be excluded to still appear. If that's happening, something is going wrong here

[Pheubel]

Strange, I've been using comments like the ones in the example in my libraries on this branch with no issue. Is there something about your comments that may differ from the examples?

As far as i can tell, my comments were structured exactly like the example. I double-checked before posting about it.

By "at this point", are you referring to the most recent stable build (9.5.2) or this PR? Because in this PR the intended behavior is to not have files that should be excluded to still appear. If that's happening, something is going wrong here

With at this point, i was referring to the state the PR was in. I did some basic testing with the ignore patterns, including what happens if the file has already been discovered by TagStudio.

I am not sure if the plan is to completely remove the entry if it is ignored or to simply not display it. But if the plan is to remove the entry when ignored, on the GUI side a warning would most likely be a good thing if there were already tags/fields attached to the entry to prevent accidental data deletion.

I hope that clears some things up.