This Repo is designed for the article that is submmitted in MIE 2024

⚠️ WARNING: This experimental project is under rapid development and lacks basic safeguards. Until a stable 1.0 release, ONLY run this repository on devices without sensitive information or access to paid services. ⚠️

Topic Extraction, also known as "automatic topic discovery" or "topic modeling," is a text analysis technique used to identify the main themes or concepts in a collection of documents or a large text body. The goal is to automatically summarize and categorize the content by extracting the most relevant keywords, phrases, or topics, thereby enabling users to better understand and explore the information. Topic extraction algorithms detect patterns and probabilistically determine topics based on the frequency and distribution of words, as well as their co-occurrence in the text. These techniques help in a variety of applications, such as sentiment analysis, content recommendation, and search optimization.

A Co-occurrence Topic Map is a visual representation of the relationships between topics, keywords, or concepts based on their co-occurrence within a collection of documents or a large text body. In a Co-occurrence Topic Map, topics that frequently appear together in the text are connected by lines, edges, or proximity, indicating a thematic or semantic relationship between the connected items. The map can be used to explore and analyze the structure and content of a text corpus, identify key themes or trends, and support navigation and knowledge discovery. This type of visualization can provide a comprehensive and intuitive overview of the data, revealing hidden patterns and enabling users to gain insights that might be difficult to discern from the raw text alone.

Clone repository:

git clone https://github.com/mjafarpour87/medical-insights.git

or

git clone [email protected]:mjafarpour87/medical-insights.git

Create environment variable:

python -m venv venv

Activate environment variable:

Windows

$ .\venv\Scripts\activate

Linux

$ source venv/bin/activate

pip install -r requirements.txt

You can install last version of triplea with this command

pip install git+https://github.com/EhsanBitaraf/triple-a.git

python -m spacy download en_core_web_sm

In step 7, a dataset is formed that can be used in the next steps and various studies can be done on it. This dataset includes the title of the article, the list of topics extracted using LLM and the topics extracted using the Textrank method, as well as the keywords of the article. In addition, this dataset can be downloaded from here. The format of the dataset is Json.

Below is the format of each article in the dataset as Json:

{
        "title": "Review of recent preclinical and clinical research on ligand-targeted liposomes as delivery systems in triple negative breast cancer therapy.",
        "year": "2024",
        "pmid": "38520185",
        "keywords": [
            "Triple negative breast cancer",
            "drug carriers",
            "ligand-targeted liposomes",
            "liposome"
        ],
        "textrank_topics": [
            "TNBC treatment",
            "progressed TNBC treatment",
            "various treatment methods",
            "targeted treatment",
            "triple negative breast cancer therapy",
            "targeted drug carriers",
            "TNBC",
            "breast cancer patients",
            "appropriate treatment",
            "drug delivery"
        ],
        "llm_topics": [
            "Triple-negative breast cancer (TNBC)",
            "Chemotherapy",
            "Targeted treatment",
            "Liposomes",
            "Drug delivery",
            "Ligand-targeted liposomes",
            "TNBC therapy",
            "Preclinical research",
            "Clinical research",
            "MDR cancer cells"
        ]
    },

If you use this dataset in another scientific work, you can refer to it as follows:

Bitaraf, Ehsan (2024). Topic Extraction Dataset. figshare. Dataset. https://doi.org/10.6084/m9.figshare.25533532

0-0

3-7

0-7

The manuscript is submitted to MIE 2024. Upon Acceptance, further details on how to cite will be provided.

Made with contrib.rocks.

Please see our contributing guidelines for more details on how to get involved.

THis Repository is available under the MIT License.