Describe the bug

If bash-completion package is installed, bash does not generate completions with $(.

To reproduce

1. Execute touch test.out in current directory.
2. Execute for i in $(cat <TAB>.
3. No completions generated for cat command.

Expected behavior

There should be completions listed for cat command.

Versions (please complete the following information)

• Operating system name/distribution and version: Fedora release 33 (Thirty Three)
• bash version, echo "$BASH_VERSION": 5.0.17(1)-release
• bash-completion version, (IFS=.; echo "${BASH_COMPLETION_VERSINFO[*]}"): 2.8

Additional context

It is caused by a behavior change introduced in bash-4.3 (by this commit to bashline.c).

I discussed this change with Chet over e-mail and this is what he mentioned:

It looks like the completer for `for' doesn't know how to handle getting
the command substitution as a single word (since that's what it is). I
guess it must be trying to parse that out on its own, using COMP_WORDS,
since you get this when you run an instrumented function as the completer:

touch /tmp/test.txt
for f in $(cat /tmp/te[TAB]

args: for /tmp/te in
declare -a COMP_WORDS=([0]="for" [1]="f" [2]="in" [3]="$(cat /tmp/te")
COMP_CWORD=3

Note that the word readline is trying to complete is always passed to the
completion function as $2, and that is /tmp/te. But the words on the line
are passed as COMP_WORDS, and ${COMP_WORDS[$COMP_CWORD]} is as above.

Bash-4.2 looked `into' the command substitution and passed

args: cat /tmp/te cat
declare -a COMP_WORDS='([0]="cat" [1]="/tmp/te")'
COMP_CWORD=1

and so ran the completion for `cat'.

But that was fragile, and didn't take into account other command
substitutions on the line. For instance, when finding the command name to
be completed, we want to skip over command substitutions in assignment
statements, something bash-4.2 had real problems with. Here's a bug report
on that:

https://lists.gnu.org/archive/html/bug-bash/2011-12/msg00053.html

So bash-4.3 changed this. The actual change was 12/16/2011 to bashline.c:
find_cmd_start(), which now considers command substitutions as single
words, or parts of single words, as the rest of the shell does. So the
words that a completer gets are closer to the words that the parser will
eventually produce. That is why bash-4.2 thinks the command name is cat' and bash-4.3 thinks it's for'.

If bash-completion has a function that understands how to complete words
beginning with `$(', possibly recursively, that could work. Or the function
could look at $2 to see the word that readline thinks should be completed
(readline's pretty generic, and just goes back to the whitespace).