Skip to content

Commit

Permalink
version 0.24.0
Browse files Browse the repository at this point in the history
  • Loading branch information
octimot committed Feb 22, 2024
1 parent ab8beb2 commit 3e52fe5
Show file tree
Hide file tree
Showing 5 changed files with 258 additions and 40 deletions.
42 changes: 42 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,48 @@

All notable changes to this project will be documented in this file, starting with version 0.17.7.

## [0.24.0] - 2024-02-22

Project Management: create projects, then link transcriptions, stories and text files to access them easier or perform project-wide searches faster.

Please back up your StoryToolkitAI configuration folder before updating to this version! The project management feature will restructure the project.json files irreversibly.


### Added
- Project Management - create and manage projects with multiple transcripts and stories
- Project Files List - view all the transcriptions, stories and other files linked to a project
- Ability to link transcriptions, stories and other files to Projects
- Ability to open transcriptions, stories and other files from the Project Files List
- Ability to find transcriptions, stories and other files by typing directly in the Project Files List
- Ability to search through all project files and synchronized NLE timeline markers using Search -> Advanced Search in current project...
- Ability to search transcriptions from the Project Files List context menu (right click on transcription)
- Ability to send full transcriptions to the Assistant from the Project Files List context menu (right click on transcription)
- Ability to copy and paste from transcription to story and from story to another story
- Ability to change transcription name and timecode data using Transcription Settings window (Edit -> Transcription Settings...)
- Metadata tab in Ingest window that allows setting timecode, frame rate, timeline/sequence name and others during ingest
- Improved navigation to project files from the Project Files List context menu (right click on file)

### Changes
- Ability to save projects independent of their NLE (e.g. Resolve) counterparts
- Removed Resolve-related buttons from main window (still available in Main Menu -> Integrations)
- Transcription SRT and TXT are no longer saved automatically when saving transcriptions
- Segments in transcriptions need to be selected to see the relevant right-click context menu for them
- Removed key 't' shortcut from main window to avoid conflicts with the find function

### Fixes
- Fixed a bug that was preventing the tool from opening transcriptions that were initially not accessible
- Added contrast to text window cursors to improve readability for light themes on macOS
- Optimized Resolve updates using observer patterns
- Fixed a bug that was opening the same text file in multiple windows
- Fixed a bug that was pointing to the wrong result in text files when multiple similar results were found in Search
- Fixed a bug that was triggering a shot removal error during video indexing
- Updates check no longer locks the app if no internet connection is available
- Fixed a bug that was causing the playhead to endlessly move backwards when syncing transcription windows with Resolve
- Transcriptions and Stories are now reloaded from disk when opened in a new window, instead of reusing them from memory
- Improved file list validation in Ingest window
- Improved Resolve render and ingest
- Other bug fixes and optimizations

## [0.23.2] - 2024-01-26

### Changes
Expand Down
58 changes: 58 additions & 0 deletions FEATURES.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,31 @@

### This page might contain references that might be outdated. Let us know if you notice them!

## Projects

Starting with version 0.24.0, the tool allows you to create and manage projects which can contain transcriptions, stories and other files.

### Creating a new project and linking files

To create a new project, click on File -> New Project... and then enter the name of the project.
Once the project is created, you should see it in the projects list in the main window when you open the tool.

By clicking on any of the projects in the list, you'll open it and see all the files that are linked to it.

Files that are linked but not accessible, will be shown in red in the main window. You can right-click on any of these files and Re-link, if you know where the file is located.

To link a file to a project, simply open the file and then click on File -> Link to Project... and then select the project you want to link the file to.
You can also automatically link new transcriptions to the current project during ingest - see the Metadata tab in the Ingest Window

Besides seeing all the files linked to a project in the main window, the main advantage of adding files to a project is that you can search through all of them in the same time by clicking the Search button in the main window or via Search -> Advanced Search in current project... in the main menu.

### Exporting and importing projects

To share projects with other users, you can export them either by clicking on the project in the projects list,
or, when inside a project by clicking on File -> Export Project... in the main menu. This will create a .zip file containing the project info files (but not the files linked to the project).

To import, click on File -> Import Project... in the main menu and then select the .zip file you want to import. Once the project is imported, you'll see all the files that need to be Re-linked in red in the main window. Right-click on any of these files and Re-link, if you know where the file is located.

## Transcriptions

### Transcription Results
Expand Down Expand Up @@ -217,6 +242,39 @@ but for now, this is a fast way to populate the transcript with speaker segments
The **Speaker Detection Threshold** allows you to tweak the speaker detection threshold between 0 and 1,
where 1 means no speaker change detection and 0 means that all segments will be assigned a different speaker.

### Ingest Metadata

Starting with version 0.24.0, we can add specific metadata that will be saved in the transcription file post-ingestion.
These main metadata fields are:

**Timeline Name** - useful when you want to reference the original timeline or sequence name in the NLE you used to render the source file.
This will also be used when Exporting Stories to EDL or XML that reference the file you're ingesting.

**Start Timecode** - the start timecode we use to calculate timecodes in the transcription.

**Frame Rate** - the frame rate we use to calculate timecodes in the transcription.

When ingesting multiple files at once, these metadata fields will be filled in either using the **Render Info File**,
or, if that's not available, the tool will try to read the metadata from the file itself.

Also from the Metadata tab in the Ingest Window: you can choose to **Delete Render Info File** after ingestion.

Just to understand how the render info file works, first it's important that it has the exact name of the file you're ingesting with the extension .json.
Then, the file should contain the following fields:

```
{
"project_name": "PROJECT NAME",
"timeline_name": "SOURCE TIMELINE NAME",
"timeline_start_tc": "HH:MM:SS:FF",
"fps": 24
}
```
When transcribing/ingesting from Resolve using StoryToolkitAI, the tool will create the Render Info File automatically,
but you can also create this programmatically or manually yourself when ingesting files from other NLEs.

If you know of similar files produced by other NLEs, please let us know so we can add support for them.

### Transcript Word Find
Once a transcript is loaded, a basic find function (CMD/CTRL+F) will let you find words in the transcript and show you
their position. Press "ENTER" to cycle between results. Once you find what you're looking for, simply clicking the
Expand Down
34 changes: 20 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,12 @@

## Description

**StoryToolkitAI is a film editing tool that tries to understand your footage and helps you edit more efficiently with
the assistance of AI.**
**StoryToolkitAI is a film editing tool that tries to understand your footage and helps you edit more efficiently with the assistance of AI.**

The tool works locally on your machine, independent of any other editing software, but it also integrates with DaVinci
Resolve Studio 18.
It transcribes, indexes scenes, helps you search through your footage, automatically selects and creates stories using large language models, like OpenAI GPT-4,
which you can then import into your editing software via EDL or XML.

The tool works locally on your machine, independent of any other editing software, but it also integrates with DaVinci Resolve Studio 18.

<img alt="StoryToolkitAI Interface" src="help/storytoolkitai_v0.19.0.png" width="750">

Expand All @@ -20,6 +21,7 @@ Resolve Studio 18.
- [X] **Translate transcripts** to other languages using OpenAI GPT (v. 0.22.0+)
- [X] **Ask AI to create Stories and Selections** based on your footage using OpenAI GPT (v. 0.22.0+)
- [X] **Automatic Speaker Detection** in transcripts (v. 0.23.0+)
- [X] **Project File Management** for more intuitive workflows and easier search
- [X] Automatic Question detection in transcripts
- [X] Transcript Groups - group transcript lines into whatever you need to find them easier
- [x] Multi-format export of transcripts, including SRT, TXT, AVID DS and as Fusion Text node
Expand All @@ -33,8 +35,8 @@ Resolve Studio 18.
- [x] Direct import of subtitles into Resolve bin

### Planned Features
- [ ] **Project File Management** for more intuitive workflows and easier search
- [ ] **Automatic Topic Classification** to help you discover ideas in your transcripts
- [ ] **Use of local/custom LLM models**
- [ ] **Integration with other AI tools**
- [ ] **Integration with other software / standalone players**
- [X] Plus more flashy features as clickbait to unrealistically raise expectations and destroy competition
Expand All @@ -48,21 +50,21 @@ For detailed features info, go [here](https://github.com/octimot/StoryToolkitAI/

To download the latest standalone release, see [the releases page](https://github.com/octimot/StoryToolkitAI/releases).

However, the standalone releases will most likely always be behind the git version, so, if you're comfortable with
using the terminal / command line and want to always have access to the newest features, we recommend that you try to
install the tool from source.
However, the standalone releases will most likely always be behind the git version,
so, if you're comfortable with the terminal / command line and want to always have access to the newest features,
we recommend that you try to install the tool from source.

For detailed installation instructions
[go here](https://github.com/octimot/StoryToolkitAI/blob/main/INSTALLATION.md).

## Is it really free?
Yes, the tool runs locally and there's no need for any additional account to transcribe or search. These features will
always be free as long as your machine supports them without external services.
Yes, the tool runs locally and there's no need for any additional account to transcribe, index video or search.
These features will always be free as long as your machine supports them without external services.

The only feature that now requires external services is the Assistant feature which relies on OpenAI ChatGPT.
The only feature that now requires external services is the Assistant which relies on OpenAI API or storytoolkit.ai API.

**Some features are released earlier only to our Patreon Patrons.** If you want to support the development,
check out our [Patreon page](https://www.patreon.com/StoryToolkitAI) and get some cool perks.
We rely on the support of our [Patreon members](https://www.patreon.com/StoryToolkitAI)!
If you want to support development and get access to new features earlier, check out our [Patreon page](https://www.patreon.com/StoryToolkitAI).

---

Expand All @@ -73,7 +75,7 @@ translate your audio.

Currently, the only features that send data from your machine to the Internet are:
- The StoryToolkitAI API Key check to storytoolkit.ai (only when entered in the Settings Window)
- The Assistant to OpenAI or storytoolkit.ai (only contexts and messages that you select and send).
- The Assistant, to OpenAI or storytoolkit.ai (only contexts and messages that you select and send).

The tool also checks for updates on every start.

Expand All @@ -98,6 +100,10 @@ The tool would be useless without using the following open source projects:
- [CustomTkinter](https://customtkinter.tomschimansky.com/)
- and many other packages listed in [requirements.txt](https://github.com/octimot/StoryToolkitAI/blob/main/requirements.txt)

## Code contributions are welcome!

Please open an issue with what you're trying to solve first and let's discuss it there.

---

# Known issues and Troubleshooting
Expand Down
Loading

0 comments on commit 3e52fe5

Please sign in to comment.