Support for URI decoding

[tapayne88]

Hi 👋

I'm wondering what your view is on supporting the URI decoding of filenames. I'm trying open filenames like this

/Users/thomas.payne/git/yarn-pnp-demo/.yarn/cache/%40testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip%3A%3Anode_modules/%40testing-library/react/types/index.d.ts

I have a little function that I can wrap these up with but it would be super handy if vim-rzip could do the URI decoding for me that way I could drop my custom URI scheme yarnpnp:

function! YarnPnPRead(uri)
    let l:parsed_uri = substitute(a:uri, '%\([a-fA-F0-9][a-fA-F0-9]\)', '\=nr2char("0x" . submatch(1))', "g")
    let l:zipfile_uri = substitute(l:parsed_uri, '^yarnpnp', 'zipfile', "g")
    call rzip#Read(l:zipfile_uri, 1)
endfunction

augroup YarnPnP
    autocmd!
    autocmd BufReadCmd yarnpnp:*,yarnpnp:*/* call YarnPnPRead(expand('<amatch>'))
augroup END

What do you think?

[lbrayner]

I had to research the terms, because I'm not familiar with OSX. By custom URI scheme you I assume you mean OSX's handling of custom protocols.

Unfortunately, Vim does not handle URI decoding, so I don't think this plugin should take such a task. There's tpope's vim-unimpaired, that has mappings for encoding and decoding URLs. Maybe that would suit you better?

[tapayne88]

Thanks for your response!

Sorry, I should have given more context. I'm trying to get the typescript language server to support yarn PnP projects in neovim using native LSP. I have a few PRs open in a few places to try to do it

• Support Yarn PnP typescript-language-server/typescript-language-server#220
• Support PnP with Neovim native LSP yarnpkg/berry#3041
• Allow URI schemes other than file: without authority neovim/neovim#14959

The tl;dr is typescript (with a bit of help from yarn) can emit file identifiers like this

zipfile:/Users/thomas.payne/git/yarn-pnp-demo/.yarn/cache/%40testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip%3A%3Anode_modules/%40testing-library/react/types/index.d.ts

which use a URI scheme zipfile. Due to the URI encoding (%40 & %3A) this file can't be opened by vim-rzip but it can if decoded.

My workaround at the minute is to have yarn set the file identifiers to yarnpnp scheme so I can URI decode them before calling your plugin

yarnpnp:/Users/thomas.payne/git/yarn-pnp-demo/.yarn/cache/%40testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip%3A%3Anode_modules/%40testing-library/react/types/index.d.ts

becomes

zipfile:/Users/thomas.payne/git/yarn-pnp-demo/.yarn/cache/@testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip::node_modules/@testing-library/react/types/index.d.ts

Your plugin already does 99% of what I need and thought it isn't too much of a stretch to include URI decoding but I very much defer to your opinion on that 😄.

I guess I could use vim-unimpaired's decode algorithm but I would still need to have a custom function / autocmd to handle the file loading.

[lbrayner]

You can hack vim-rzip with the help of the VimEnter event. I think I was able to to solve your problem with this:

function! ParseURI(uri)
    return substitute(a:uri, '%\([a-fA-F0-9][a-fA-F0-9]\)', '\=nr2char("0x" . submatch(1))', "g")
endfunction

function! RzipOverride()
    autocmd! zip BufReadCmd   zipfile:*/*
    exe "au! zip BufReadCmd ".g:zipPlugin_ext
    autocmd zip BufReadCmd   zipfile:*/* call rzip#Read(ParseURI(expand("<amatch>")), 1)
    exe "au zip BufReadCmd ".g:zipPlugin_ext." call rzip#Browse(ParseURI(expand('<amatch>')))"
endfunction

autocmd VimEnter * call RzipOverride()

I tested it with a file named nested.zip inside @zipinzip.zip, with the ex command e zipfile:/home/desenvolvedor/Downloads/\%40zipinzip.zip::nested.zip, the contents of which I was able to edit.

See if that works for you!

[lbrayner]

As I was testing, I forgot to replace the colons with 3A (yielding e zipfile:/home/desenvolvedor/Downloads/\%40zipinzip.zip%3A%3Anested.zip). When I did, the write command failed! I'll look into that soon.

[tapayne88]

Ah, that's awesome!

One thing I'm having difficultly with is that URI encoding uses % which vim seems to expand with in expand("<amatch>") calls to the file path. Do you know a way to prevent this?

[lbrayner]

Ah, that's awesome!

One thing I'm having difficultly with is that URI encoding uses % which vim seems to expand with in expand("<amatch>") calls to the file path. Do you know a way to prevent this?

Could you give me an example of you're trying to do? For example, if it's an ex command, such as e zipfile:/home/desenvolvedor/Downloads/\%40zipinzip.zip::nested.zip, note that the percent sign is escaped. If it's an evaluated expression, such as exe "e " . fnameescape('zipfile:/home/desenvolvedor/Downloads/%40hi.zip%3A%3Anested.zip'), you can use fnameescape.

[tapayne88]

So I think I might have been misunderstanding what was happening.

I've been using this command to debug

:exe "e ".fnameescape("zipfile:/Users/thomas.payne/git/github.com/yarn-pnp-demo/.yarn/cache/%40testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip%3A%3Anode_modules/%40testing-library/react/types/index.d.ts")

When listing buffers (:ls) I see something like

  2 %a   "zipfile:/Users/thomas.payne/git/github.com/yarn-pnp-demo/.yarn/cache/zipfile:/Users/thomas.payne/git/github.com/yarn-pnp-demo/.yarn/cache/%40testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip%3A%3Anode_modules/%40testing-l
ibrary/react/types/index.d.ts40testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zipzipfile:/Users/thomas.payne/git/github.com/yarn-pnp-demo/.yarn/cache/%40testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip%3A%3Anode_modules/%4
0testing-library/react/types/index.d.ts3Azipfile:/Users/thomas.payne/git/github.com/yarn-pnp-demo/.yarn/cache/%40testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip%3A%3Anode_modules/%40testing-library/react/types/index.d.ts3Anode_mo
dules/zipfile:/Users/thomas.payne/git/github.com/yarn-pnp-demo/.yarn/cache/%40testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip%3A%3Anode_modules/%40testing-library/react/types/index.d.ts40testing-library/react/types/index.d.ts" line 1

I think these 2 lines are causing this issue. Do you think we could replace fn with a:fname?

vim-rzip/autoload/rzip.vim

Line 67 in b67c1a5

let fn = expand('%:p')

vim-rzip/autoload/rzip.vim

Line 73 in b67c1a5

sil exe 'keepalt file '.fnameescape(fn)

[tapayne88]

This repo might be of use with that above
https://github.com/tapayne88/yarn-pnp-demo

[lbrayner]

I think these 2 lines are causing this issue. Do you think we could replace fn with a:fname?

These lines are in upstream, but don't worry, I found a solution. The trick is the order in which autocommands are added. One needed only to execute exe "keepalt file " . fnameescape(ParseURI(expand("<amatch>"))) before doing call rzip#Read(expand("<amatch>"), 1), which is in the original autocommand, like so:

function! ParseURI(uri)
    return substitute(a:uri, '%\([a-fA-F0-9][a-fA-F0-9]\)', '\=nr2char("0x" . submatch(1))', "g")
endfunction

function! RzipOverride()
    autocmd! zip BufReadCmd   zipfile:*/*
    exe "au! zip BufReadCmd ".g:zipPlugin_ext
    autocmd zip BufReadCmd   zipfile:*/* exe "keepalt file " . fnameescape(ParseURI(expand("<amatch>")))
    autocmd zip BufReadCmd   zipfile:*/* call rzip#Read(expand("<amatch>"), 1)
    exe "au zip BufReadCmd ".g:zipPlugin_ext." call rzip#Browse(ParseURI(expand('<amatch>')))"
endfunction

autocmd VimEnter * call RzipOverride()

I just quickly tested this with much success. See if works for you.

[tapayne88]

It almost works 😄 Found the rzip#Read call still needs to use ParseURI but otherwise it works!

function! ParseURI(uri)
    return substitute(a:uri, '%\([a-fA-F0-9][a-fA-F0-9]\)', '\=nr2char("0x" . submatch(1))', "g")
endfunction

function! RzipOverride()
    autocmd! zip BufReadCmd zipfile:*,zipfile:*/*
    exe "au! zip BufReadCmd ".g:zipPlugin_ext

    autocmd zip BufReadCmd zipfile:*,zipfile:*/* exe "keepalt file " . fnameescape(ParseURI(expand("<amatch>")))
    autocmd zip BufReadCmd zipfile:*,zipfile:*/* call rzip#Read(ParseURI(expand('<amatch>')), 1)
    exe "au zip BufReadCmd ".g:zipPlugin_ext." call rzip#Browse(ParseURI(expand('<amatch>')))"
endfunction

autocmd VimEnter * call RzipOverride()

I think the last thing I'm struggling (and I'm not sure it's vim-rzip's problem) is other bits of code trying to load the same file multiple times, it causes an error Buffer with this name already exists. Neovim's native LSP does it when trying to open a definition location (here). This function has already opened the file in a buffer (under the parsed named) however when it tries to load the filename again (prior to URI deocding) our autocommand kicks in and tries rename to the already open buffer - hence the error.

I'm not sure how to handle instance like this as we only rename the buffer when it's read 🤔

[lbrayner]

other bits of code trying to load the same file multiple times, it causes an error Buffer with this name already exists

Yes, this is a shortcoming of the bundled in zip.vim, which I did not address in vim-rzip. Both vanilla zip.vim and gzip.vim's supported use case was just quickly browsing an archive (and possibly even editing a text file in it) with Vim by running vim archive.zip from the command and then closing it. If you run a long session you will certainly encounter bugs.

If I think of something I'll let you know, since this would be very useful to me, because I run long and persistent sessions with the help of vim-obsession. I haven't started using LSPs yet, but hope to soon.

[tapayne88]

Cool, thanks so much for your help! I'm building up my vim understanding and I really appreciate you taking the time to answer my questions and help debug!

Do you think the above autocommands make sense to be incorporated into vim-rzip behind a feature switch or perhaps in the readme?

let g:rzip_enable_uri_parsing = v:true

[lbrayner]

Do you think the above autocommands make sense to be incorporated into vim-rzip behind a feature switch or perhaps in the
readme?

I'd have to think a little more about it.

other bits of code trying to load the same file multiple times, it causes an
error Buffer with this name already exists.

Part of the solution would be changing this line:

vim-rzip/autoload/rzip.vim

Line 288 in b67c1a5

setlocal bufhidden=hide

into

setlocal bufhidden=wipe

This will wipe the buffer as soon as it's hidden, thus avoiding Buffer with this name already exists.

But there's the case when the buffer is still being displayed, i.e. not
hidden — perhaps in another tab. It has to be wiped or focused, which implies
switching windows or possibly even tabs. What do you think? Do you think that's
too heavy handed?

[tapayne88]

I'm not sure that will help because I think the error is thrown when we attempt to rename the buffer to the parsed name. Neovim internals are all handling filenames like this

/Users/thomas.payne/git/github.com/yarn-pnp-demo/.yarn/cache/%40testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip%3A%3Anode_modules/%40testing-library/react/types/index.d.ts

They look for already open buffers with that name - in some scenarios there may be one but it'll likely be empty (or unloaded. Neovim will try to create it and when our autocommand runs it'll rename it causing the error.

I think it might help in the scenario where neovim finds a matching buffer (incorrect one) and loads it, this will show an empty buffer.

I was wondering if we could in the BufReadCmd autocommand detect any open buffers with our target name and return that rather than opening a new one.

[lbrayner]

Since I'm not currently using Neovim's LSP features, I'm a bit confused as to the complete picture of your use case.

I'm not sure that will help because I think the error is thrown when we attempt to rename the buffer to the parsed name. Neovim internals are all handling filenames like this

/Users/thomas.payne/git/github.com/yarn-pnp-demo/.yarn/cache/%40testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip%3A%3Anode_modules/%40testing-library/react/types/index.d.ts

So Neovim's internals are handling file names like that. Do you mean a URI such as file:///Users/thomas.payne/git/github.com/yarn-pnp-demo/.yarn/cache/%40testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip%3A%3Anode_modules/%40testing-library/react/types/index.d.ts? Because it doesn't make sense to me if it's only /Users/thomas.payne/git/github.com/yarn-pnp-demo/.yarn/cache/%40testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip%3A%3Anode_modules/%40testing-library/react/types/index.d.ts without some protocol (like file://).

From this answer:

The tl;dr is typescript (with a bit of help from yarn) can emit file identifiers like this

zipfile:/Users/thomas.payne/git/yarn-pnp-demo/.yarn/cache/%40testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip%3A%3Anode_modules/%40testing-library/react/types/index.d.ts

From that point on, I thought your issue was limited to URI's with the zipfile: protocol.