Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

disable/remove jekyll related posts (--lsi) + related dependencies #114

Merged
merged 2 commits into from
Jul 3, 2024

Conversation

0xdevalias
Copy link
Owner

This PR comments out the flags/dependencies related to Jekyll's 'related posts' feature, and specifically the --lsi 'Latent Semantic Indexing' feature for calculating related posts based on similarity.

It was pointed out that we're not even currently making uses of the related posts functionality, so the complexities of trying to get the dependencies to speed up --lsi aren't even relevant to us currently:

A few notes that you might find helpful:

  • You're not noticing any difference in build times with the --lsi option because your site (as it is today in this repo) doesn't use related posts (so the --lsi option does nothing). To use LSI, you need to call site.related_posts somewhere in a Liquid template. For example, you might add something like the following to _layouts/post.html:
    {% for post in site.related_posts limit:3 %}
      <p>{{ post.title }}</p>
    {% endfor %}
  • When you call site.related_posts, if you don't pass the --lsi option, it's just recent posts.
  • If you are using site.related_posts and you pass the --lsi option, You'll see Populating LSI... in your jekyll build --lsi output. The build will be slow unless you have the gsl gem and native gsl library installed. I haven't experimented with nmatrix or narray at all, but simply using the gsl gem results in a ~500x speed increase for my use.

Originally posted by @mkasberg in #83 (comment)

That said, it would be nice to bring back some kind of 'related posts' feature at some point.. and there are some further notes about that in the following comment (both with using --lsi + more updated dependencies, or by using OpenAI embeddings/similar):

See Also

This PR partially fixes/relates to the following:

And may have some impact on the following:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant