[Request] 能够定义只发送图片时的填充prompt

[1PercentSync]

🥰 Feature Description

目前的视觉模型没法只发送图片，然而进行重复性任务的时候，比如截图粘贴然后要求回答问题，手动打字很浪费时间，复制粘贴也由于剪切板中有图片被覆盖

🧐 Proposed Solution

如果能添加一个设置选项，设置一个prompt，比如“识别图片中文字，然后回答问题”
在文本为空的时候填充文本然后和图片一起发出去就能提高效率了

📝 Additional Information

No response

[lobehubbot]

👀 @1PercentSync

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible.
Please make sure you have given us as much context as possible.
非常感谢您提交 issue。我们会尽快调查此事，并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

🥰 Feature Description

The current visual model cannot only send pictures. However, when performing repetitive tasks, such as taking screenshots and pasting them and then asking to answer questions, manual typing is a waste of time. Copying and pasting is also overwritten because there are pictures in the clipboard.

🧐 Proposed Solution

If you can add a setting option, set a prompt, such as "Recognize the text in the picture and then answer the question"
Filling the text when it is empty and then sending it together with the picture can improve efficiency.

📝 Additional Information

No response

[1PercentSync]

对于gpt4v，可以填充一个空格，对于claude3，prompt只有空格的话会报text content blocks must contain non-whitespace text，可以使用一个:

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

For gpt4v, you can fill in a space. For claude3, if the prompt only has spaces, it will report text content blocks must contain non-whitespace text. You can use one:

[1PercentSync]

然后剩下的交给system prompt处理，这是目前情况下最高效的办法

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Then leave the rest to the system prompt. This is the most efficient method under the current circumstances.

[arvinxx]

试下助手设置里的inputTemplate

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Try the inputTemplate in the assistant settings

[1PercentSync]

试下助手设置里的 inputTemplate

不行，文本框为空的时候没法发送，能否把检测空的步骤移到inputTemplate合成之后

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Try the inputTemplate in the assistant settings

No, the text box cannot be sent when it is empty. Can the step of detecting the empty space be moved after the inputTemplate is synthesized?

[arvinxx]

这个检测我优化下，做成如果有图片的情况下允许字符串内容为空这样

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

I optimized this detection to allow the string content to be empty if there is a picture.

[lobehubbot]

✅ @1PercentSync

This issue is closed, If you have any questions, you can comment and reply.
此问题已经关闭。如果您有任何问题，可以留言并回复。

[lobehubbot]

🎉 This issue has been resolved in version 1.0.14 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀