Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: openai/tiktoken
base: main
Choose a base ref
...
head repository: Miuler/tiktoken
compare: main
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 2 commits
  • 15 files changed
  • 1 contributor

Commits on Sep 15, 2023

  1. ✨ feat: Split code rust, for run as python lib and rust lib

    @Miuler
    Miuler committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    b7af705 View commit details
    Browse the repository at this point in the history

  2. ✨ feat(rust): Convert project to a multi-crate workspace 

    This commit restructures the project from a single-crate workspace into a multi-crate workspace, dividing it into 'rs-tiktoken' and 'py-tiktoken'. This is done to improve the clarity of the organization of the codebase and make the Rust and Python modules separate for easier code maintenance. The setup.py is also updated to reflect these changes in the directory structure.

Refs: #24
    @Miuler
    Miuler committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    6b7624a View commit details
    Browse the repository at this point in the history