Question about learnable text #88
Description
Hello, author! Your article is excellent and I've learned a lot. But in the method part, the prefix is [0, :1, …], the suffix is [0, 1 + n_ctx:, :], so are the original text embeddings from 1 to n_ctx directly replaced by the learnable text? Won't the original part be used anymore?
Also ,in
x = torch.cat([prefix, textual_context, suffix], dim=0)
would the textual_context replace the middle part again?
And the other question is that IF I SET the initial_prompt as "X X X X X X X X [class]", I would get the tokenized prompt which size is [1, 77] and final turn to [1, 77, 768] after embedding layer. Would the suffix of this prompt never change during training stage? In this case, I think I should set a better initial_prompts to earn higher performance.
Finally, thanks to your awesome work again! :)