Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support including file attachments in the chat message #957

Merged
merged 46 commits into from
Nov 11, 2024
Merged
Changes from 1 commit
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
362bdeb
Add methods for reading full files by name and including context
sabaimran Nov 5, 2024
a27b8d3
Remove summarize condition for only 1 file filter
sabaimran Nov 5, 2024
3dc9139
Add additional handling for when file_object comes back empty
sabaimran Nov 5, 2024
7543360
Merge branch 'master' of github.com:khoj-ai/khoj into features/includ…
sabaimran Nov 5, 2024
1f372bf
Update file summarization unit tests now that multiple files are allowed
sabaimran Nov 5, 2024
cf0bcec
Revert SKIP_TESTS flag in offline chat director tests
sabaimran Nov 5, 2024
dc26da0
Add uploaded files in the conversation file filter for a new convo
sabaimran Nov 5, 2024
a0480d5
use fill weight for the toggle right (enabled state) for research mode
sabaimran Nov 5, 2024
de73cbc
Add support for relaying attached files through backend calls to models
sabaimran Nov 7, 2024
3b1e846
Include attach files in calls to extract questions
sabaimran Nov 7, 2024
3940351
Add an api that gets a document, and converts it to just text
sabaimran Nov 8, 2024
ecc81e0
Add separate methods for docx and pdf files to just convert files to …
sabaimran Nov 8, 2024
b8ed985
Accept attached files in the chat API
sabaimran Nov 8, 2024
140c67f
Remove focus ring from the text area component
sabaimran Nov 8, 2024
92c3b9c
Add function to get an icon from a file type
sabaimran Nov 8, 2024
e521853
Remove unnecessary console.log statements
sabaimran Nov 8, 2024
a89160e
Add support for converting an attached doc and chatting with it
sabaimran Nov 8, 2024
3a51996
Process attached files in the chat history and add them to the chat m…
sabaimran Nov 8, 2024
9bbe27f
Set default value of attached files to empty list
sabaimran Nov 8, 2024
4b8be55
Convert UUID to string when forking a conversation
sabaimran Nov 8, 2024
33498d8
Simplify the share chat page. Don't need it to maintain its own conve…
sabaimran Nov 8, 2024
623a97a
Merge branch 'master' of github.com:khoj-ai/khoj into features/includ…
sabaimran Nov 8, 2024
ee062d1
Fix parsing for PDFs via content indexing API
sabaimran Nov 8, 2024
ad46b0e
Label pages when extract text from pdf, docs content. Fix scroll area…
sabaimran Nov 8, 2024
4695174
Add support for file preview in the chat input area (before message s…
sabaimran Nov 8, 2024
7159b0b
Enforce limits on file size when converting to text
sabaimran Nov 8, 2024
807687a
Automatically generate titles for conversations from history
sabaimran Nov 9, 2024
3badb27
Remove stored uploaded files after they're processed.
sabaimran Nov 9, 2024
bc95a99
Make tracer the last input parameter for all the relevant chat helper…
sabaimran Nov 10, 2024
e5ac076
Move construct_chat_history method back to conversation.utils.py
sabaimran Nov 10, 2024
dbf0c26
Remove _summary_ description in function descriptions
sabaimran Nov 10, 2024
459318b
And random suffixes to decreases any clash probability when writing t…
sabaimran Nov 10, 2024
835fa80
Allow docx conversion in the chatFunction.ts
sabaimran Nov 10, 2024
92b6b3e
Add attached files to latest structured message in chat ml format
sabaimran Nov 10, 2024
bd55028
Fix randint import from random when creating filenames for tmp
sabaimran Nov 10, 2024
79b15e4
Only add images when they're present and vision enabled
sabaimran Nov 10, 2024
2c543be
Add typing to the constructed messages listed
sabaimran Nov 10, 2024
170d959
Handle offline messages differently, as they don't respond well to th…
sabaimran Nov 10, 2024
55200be
Apply agent color fill to the toggle both in off and on states
sabaimran Nov 11, 2024
8805e73
Merge branch 'master' of github.com:khoj-ai/khoj into features/includ…
sabaimran Nov 11, 2024
fd15fc1
Move construct chat history back to it's original position in file
debanjum Nov 11, 2024
4223b35
Use python stdlib methods to write pdf, docx to temp files for loaders
debanjum Nov 11, 2024
7954f39
Use accept param to file input to indicate supported file types in we…
debanjum Nov 11, 2024
ae4eb96
Consolidate file name to icon mapping
sabaimran Nov 11, 2024
47937d5
Merge branch 'features/include-full-file-in-convo-with-filter' of git…
sabaimran Nov 11, 2024
2bb2ff2
Rename attached_files to query_files. Update relevant backend and cli…
sabaimran Nov 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Include attach files in calls to extract questions
  • Loading branch information
sabaimran committed Nov 7, 2024
commit 3b1e8462cd6daf60216c679e76d617efac1932a2
5 changes: 5 additions & 0 deletions src/khoj/routers/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -351,6 +351,7 @@ async def extract_references_and_questions(
query_images: Optional[List[str]] = None,
agent: Agent = None,
tracer: dict = {},
attached_files: str = None,
):
user = request.user.object if request.user.is_authenticated else None

Expand Down Expand Up @@ -425,6 +426,7 @@ async def extract_references_and_questions(
max_prompt_size=conversation_config.max_prompt_size,
personality_context=personality_context,
tracer=tracer,
attached_files=attached_files,
)
elif conversation_config.model_type == ChatModelOptions.ModelType.OPENAI:
openai_chat_config = conversation_config.openai_config
Expand All @@ -443,6 +445,7 @@ async def extract_references_and_questions(
vision_enabled=vision_enabled,
personality_context=personality_context,
tracer=tracer,
attached_files=attached_files,
)
elif conversation_config.model_type == ChatModelOptions.ModelType.ANTHROPIC:
api_key = conversation_config.openai_config.api_key
Expand All @@ -458,6 +461,7 @@ async def extract_references_and_questions(
vision_enabled=vision_enabled,
personality_context=personality_context,
tracer=tracer,
attached_files=attached_files,
)
elif conversation_config.model_type == ChatModelOptions.ModelType.GOOGLE:
api_key = conversation_config.openai_config.api_key
Expand All @@ -474,6 +478,7 @@ async def extract_references_and_questions(
vision_enabled=vision_enabled,
personality_context=personality_context,
tracer=tracer,
attached_files=attached_files,
)

# Collate search results as context for GPT
Expand Down