Skip to content

Felliks/Neurite

 
 

Repository files navigation

Neurite

License: MIT

localhost_8080_ (9)

Introduction

💡 Neurite unleashes a new dimension of digital interface...

...the fractal dimension...

Bridging Fractals and Thought

Inspired by chaos theory and graph theory, Neurite unveils the hidden patterns and intricate interconnections that shape creative thinking. To do this, we developed a virtually limitless fractal workspace that blends the mesmerizing complexity of the Mandelbrot set with contemporary mind mapping techniques.

Why?

Fractals are not just an aesthetic choice, their inner logic is ingrained into a number of natural and constructed phenomena - ranging from polynomial equations, to art, and even the cosmic web.

Neurite is unique in its ability to display the Mandelbrot set in real time. Movements through the fractal are no longer interrupted at each step. Instead, the fractal generation persists and regenerates as you interact!

Nodes represent text, images, videos, code, and AI agents. Together, they thread a personalized microcosm of your thoughts and inspirations.

Complexity Inspires Creativity

Whether you're a researcher, writer, artist, developer, educator, or simply a curious thinker, Neurite offers a departure from the conventional interface. It is a multifaceted landscape for ever branching pathways of thought.

Available through Github Pages: Neurite

⚠️ Warning: Contains zooming, flashing lights, and colors which may not currently be suitable for people with photosensitive epilepsy.

See the User Guide for instructions on running locally

🌱 This is an open-source project in active development, we are looking to grow our team! 🚧

feel free to join our Discord!

Table of Contents

  1. Introduction
  2. Synchronized Knowledge Management
  3. How to Use Neurite
  4. FractalGPT
  5. Neural API
  6. Local Ai
  7. User Guide
  8. Local Server Setup
  9. Future of Neurite
  10. Contact

Synchronized Knowledge Management

Neurite provides markdown formatted, bi-directional synchronization between UI-focused Mind-Mapping and text-based hyperlinking (Zettelkasten).

This versatile approach allows one to seamlessly organize thoughts without sacrificing the benefits of either technique.

notepanel

  • Bi-Directional Sync: Edit a Mind Map node's content and see it instantaneously update in the Zettelkasten. Conversely, any text changes in the Zettelkasten are reflected by the Mind Map.
  • Dynamic Fractal Backdrop: The Mandelbrot fractal is not just an aesthetic element - fractals set the stage for an interactive open-world landscape of multi-media composition.
  • Zoom-to-Node: Navigate through your notes from within the Zettelkasten, and zoom directly to their corresponding nodes within the Mandelbrot set. Conversely, clicking the fullscreen of a text node navigates to the corresponding note in the Zettelkasten.

Build your Zettelkasten through UI interactions in the Mind Map, and reciprocally shape the Mind Map through text-based note-taking in the Zettelkasten. This navigational fluidity offers unprecedented control over both the granular and macroscopic perspectives of your information.

Custom node: and ref: tags can be set in the ? tab of the main dropdown.

How to Use Neurite

📚 Neurite lets you embed anything - text, photos, audio, video, PDFs, AI, and webpages - directly into the Mandelbrot set.

Drag and drop local files or web content straight into the fractal.

Supports embedded links. Ctrl+V to paste an embed link, or pasting normal links embeds them into an iframe if supported by the host.

  • Talk to Neurite GPT with access to Neurite's code and readme.

Controls

*Full controls are listed in the ? tab within Neurite's main dropdown.

Scroll to zoom through the fractal.

Click and drag to move around

Shift + double click to create nodes.

Shift to freeze nodes.

Double click to anchor node.

Hold Shift + click two nodes to connect.

Alt + double click to create an Ai node.

Alt + drag textareas to arrange the node rather than highlighting text.

Or, drag in nodes from the dropdown.

UI Tips

  • Make sure to zoom (Scroll) deeper into the fractal before creating nodes. Otherwise, nodes will not have any space to spread out and become too big relative to the fractal.
  • When arranging windows, drag them via the title input. Rescale by holding Shift and scrolling over the node.
  • Use Shift to freeze nodes in place, then double click the nodes that should remain anchored once unfrozen (releasing Shift.)
  • The AI tab has controls for token limit, temperature, number of searched nodes sent as context, and length of recent conversation sent. (You may need to turn up the token count for longer conversations.)
  • Connect instructions to Ai nodes. For every message, any connected nodes will be sent to the Ai. (ie, the entire connected graph)

FractalGPT

Not your typical chat interface

  • Web of Thought Reasoning: Engage with non-linear, rhizomatic memory through our biomimetic interface.
  • Limitless AI Agents: Create and connect an endless number of individual chat windows, or let the AI take notes from above.
  • Fractal Interface: Utilize a custom fractal environment designed for Iterative Exploratory Learning.
  • Custom Tool Creation Connect nodes that contain code blocks together to run html/css/js in an iframe, or Python via Pyodide.
  • Neurite Function Calling Call Neurite functions from within Neurite, and let the Ai take full control with context from the newly released Neural API

Modular Conversation

In Neurite, you don't just interact with AI; you grow its contextual memory, thanks to:

  • Structured Memory Mapping: Anchor the AI's thought process as notes within an evolving fractal mind map.
  • Graph your AI Conversations: Segment AI's responses into a cohesive web of thought that both improves AI reasoning and allows for the history and chains of your conversation to dynamically construct the Ai's memory.
  • Interactive Fractal Network: Neurite's fractal interface acts as a personal, emergent, and contextually aware database of knowledge.
  • Intuitive Note Navigation: Effortlessly access and pull your notes into view, no matter where they are within the fractal landscape.

Unbounded AI Exploration

An open world generative landscape for thought integrated with artificial intelligence.

Connect to the AI using an OpenAI API key or from directly within your browser via webLLM. Let the Mandelbrot set shape the endless topology for your mind-map, guiding it into an interconnected tapestry of thoughts, references, and creativity.

Sierpinski.mov

Neurite Supports the Following AI Plugins:

  • Auto Mode: Enable the AI to recursively generate its own prompts.
  • Node-Based Memory: Utilize long-term recall through our vector-embedded search of your notes and conversation. (Experimental forgetting mode in the ? tab)
  • Direct Rendering Environments:
    • HTML/JS: Render GPT's code output directly. Connected nodes bundle together any HTML/CSS/JS codeblocks or editors.
    • Python (Pyodide): Execute Python code directly within the browser.
  • Web Search: Utilize the Google programmable search API to retrieve webpages that can be sent as context to the AI.
  • Webpage and PDF Text Extraction: Leverage a local vector database to extend the AI's context window.
  • Wikipedia Results: Retrieve the top 3 Wikipedia results or shuffle through the top 20.
  • Wolfram Alpha Results: Experience our unique implementation of integrating Wolfram with AI.

All API keys can be input through the AI tab in the menu dropdown.

Neural API

movementapi

The newest update to Neurite is our function calling panel.

This feature is terminal that allows you to execute Neurite's code from within Neurite itself.

Included with the function calling panel update is our Neural API. The Neural API is a growing collection of existing features within Neurite, made for sequencing animations of function calls. The current features include,

  • Animate movements through the Mandelbrot set
  • Determine exact coordinates to zoom to
  • Call GPT 4 Vision to determine the next movement. (set number of iterations)
  • Create notes
  • Prompt the Zettelkasten Ai
  • Prompt the user
  • Search and Zoom to Notes

You can call on GPT to construct the sequences of function calls itself. It has access to the documentation.

All of these features can be called as a sequence or as decided by your code.

For the current documentation, try calling the below from within the function panel...

const neuralAPI = neuralApiPrompt();
console.log(neuralAPI);

... the current Neural API documentation will display in your call history for the function calling panel as well as in your browser console.

There will be more information on this soon. This is a new feature in initial release.

Local Ai

LocalAi.mov

Above is an example of conversation history being shared between GPT-4 and a Local Ai model, Red Pajamas, run through webLLM.

Ai Nodes allow for the creation of an arbitrary number of Ai Agents with the ability to connect memories together.

  • Alt + Double click creates a node with a more traditional AI chat interface. (you can create multiple)
  • Connected Ai nodes will now send messages to one another. This has been improved but contains endless possiblities. (Hold Shift and Click two node windows to connect them together)
  • Supports Local and OpenAi models concurrently.

Inter-AI Communication

  • Message Looping: Initiate conversational loops among connected AI nodes.
  • Prompt Generation: Upon connection, AI nodes are programmed to end responses with a question or prompt, which is then sent to any other connected nodes.
  • Unique & Shared Instructions:
    • Unique Instructions: Connect text nodes exclusively to a specific AI node to give it private system instructions, isolated from other connected AI nodes.
    • Shared Instructions: Text nodes can send instructions to multiple AI nodes as long as there are no other AI nodes between them. This is because...
      • Context Cut-Off: ...once another AI node is found in the connected graph, all preceding connected nodes are excluded from the current AI's instruction set.
  • Related Research: Conversational Swarm Intelligence Paper

Neurite2

Neurite currently supports OpenAi, LLama 2, Red Pajamas 3B, and Vicuna 7B.

  • webLLM runs Ai models directly in browser on the GPU or CPU.
  • We are also running an embeddings model locally through Transformers.js for local webpage/pdf extraction.
  • This is an early implementation.

See webLLM's documentation for how to run the models on your GPU.

Wolfram.mov

Above is an example of combining the Wolfram Alpha API with GPT-4.

Wolfram, Wikipedia, and Webpage/PDF vector database plugins are accessed through the localhost servers.

User Guide

To run Neurite locally,

either:

  1. Clone the repository
  2. Open the index.html file in your browser (note: local AI features will not be enabled)

Or, to enable local AI features without using our Github pages host above:

  1. Navigate to the cloned directory in your terminal (either main or local-ai branch)
  2. Install any necessary global dependencies when running the below, (Node.js, Rust, Emuscripten, Strawberry Perl, Vite)
npm install
npm run build
npm run start

Local Server Setup

Download Localhost Servers here.

The servers enable plugin integration for webpage/pdf text extraction, Wolfram Alpha, and Wikipedia.

  • Our Wikipedia, Wolfram, and web scrape servers require Python and Node.js
  • Without any servers running, Wolfram, Wiki, and Webpage extractions will not function.
  • All servers can be run with one command. Download the Localhost Servers folder, or find it in your cloned directory. Once navigated to the Localhost Servers folder in your command line, run
  node start_servers.js

This will run all of the servers at once. They'll work on our Github pages host of Neurite, or any other way you locally access the interface.

In the '?' tab, the AI HOW-TO checkbox will send a context message to the AI that allows it to answer questions about Neurite.

Notes

  • Search: Our distinctive search mechanism transports you through your intricate web of thoughts, directly zooming to the node you are looking for. The navigational experience isn't just a gimmick; it also helps to reinforce the spatial memory of where that piece of information resides within the larger framework.
  • Saving: Your networks can be saved within the browser cache, or downloaded as a .txt file. Drag the .txt file into the save box, click load on the imported file, then click the second load button to display the network. This is still in an early state.
  • Zettelkasten and Mind Mapping: Traditional note-taking confines you to static files, while conventional mind mapping lets you visualize connections but often lacks in-depth content. Neurite treats nodes as dynamic objects that can be worked with from a variety of different scales and interfaces. Zettelkasten is a method of taking short, atomic notes (or "zettels") that link to each other to create an interconnected system. In Neurite, you don't have to choose between either approach. Create nodes through custom-defined syntax, or create nodes through UI interaction within our fractal workspace, who's contents are bi-directionally linked to the main Zettelkasten document.

Neurite realizes tapestry of thought reasoning for Ai, all within our innovative real-time Mandelbrot visualization.

Aidiagram2_1 2 1

  • This diagram represents how the AI's response incorporates note-taking to retain a non-linear conversation history.
  • We feed both the recent conversation as well as any matched notes that are not already in the recent conversation as part of the context message to the Ai.
  • This means retaining relevant information from any part of the conversation regardless of the Ai's context window size.
  • Still, advancements in context window size only further enable Neurite's performance.

Future of Neurite

Neurite is a recursive environment for generating ideas. The project combines fractals, mind-mapping, and ai to enable an experimental, yet powerful playground for research, code generation, writing, learning, and visualizing thought.

🚧 Neurite is in active development. 🚧

✅ webLLM

🔄 python LLM localhost

✅ local embeddings

  • VR
  • deeper fractal zoom

✅ Global Function Calling

✅ Global Function Calling

🔄 Ai cursor control

✅ color selection

✅ adjust fractal generation (width, length, max lines, speed)

✅ screen recording

🔄 accessibility

✅ auto embed formats

  • custom equations for the fractal
  • drawing-tool

🔄 user feedback/bug fixes

🚧