Skip to content

Commit

Permalink
feat: scraper updates and action trigger (#5)
Browse files Browse the repository at this point in the history
  • Loading branch information
akosbalasko committed Aug 31, 2021
1 parent 33bcb1f commit 7555fce
Show file tree
Hide file tree
Showing 4 changed files with 14 additions and 10 deletions.
4 changes: 4 additions & 0 deletions .github/workflows/scrape-scripts.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
name: scrape-scripts
on:
schedule:
- cron: '0 * * * *'
push:
branches:
- action-test-branch
Expand All @@ -25,4 +27,6 @@ jobs:
postprocess: ./scraper-post-process.ts
- name: Create Pull Request
uses: peter-evans/create-pull-request@v3
with:
branch-suffix: timestamp

1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ It's a common place for all of the awesome scripts developed on top of Obisidian

Feel free to share your own by creating a Pull Request with you code!


12 changes: 6 additions & 6 deletions dataview-plugin-snippet-showcase.html
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,9 @@
<meta property="og:description" content="A place to find and post code snippets used in the Dataview plugin created by Discord user blacksmithgu#1946. Edit: In a recent update, various functions have been introduced! Checkout the docs to learn more about their usage. Edit: I gave a live Obsidian Talk on the basics of Dataview, as well as some more advanced stuff. You can find the recordings of the live session here: Part 1: Metadata and Basic-to-Intermediate Dataview Part 2: More advanced Dataview stuff" />
<meta name="twitter:description" content="A place to find and post code snippets used in the Dataview plugin created by Discord user blacksmithgu#1946. Edit: In a recent update, various functions have been introduced! Checkout the docs to learn more about their usage. Edit: I gave a live Obsidian Talk on the basics of Dataview, as well as some more advanced stuff. You can find the recordings of the live session here: Part 1: Metadata and Basic-to-Intermediate Dataview Part 2: More advanced Dataview stuff" />
<meta name="twitter:label1" value="Reading time" />
<meta name="twitter:data1" value="51 mins 🕑" />
<meta name="twitter:data1" value="54 mins 🕑" />
<meta name="twitter:label2" value="Likes" />
<meta name="twitter:data2" value="683" />
<meta name="twitter:data2" value="737" />
<meta property="article:published_time" content="2021-02-26T06:12:05+00:00" />
<meta property="og:ignore_canonical" content="true" />

Expand Down Expand Up @@ -117,8 +117,8 @@ <h1>

<div itemprop="interactionStatistic" itemscope itemtype="http:https://schema.org/InteractionCounter">
<meta itemprop="interactionType" content="http:https://schema.org/LikeAction"/>
<meta itemprop="userInteractionCount" content="63" />
<span class='post-likes'>63 Likes</span>
<meta itemprop="userInteractionCount" content="66" />
<span class='post-likes'>66 Likes</span>
</div>

<div itemprop="interactionStatistic" itemscope itemtype="http:https://schema.org/InteractionCounter">
Expand Down Expand Up @@ -813,8 +813,8 @@ <h1>

<div itemprop="interactionStatistic" itemscope itemtype="http:https://schema.org/InteractionCounter">
<meta itemprop="interactionType" content="http:https://schema.org/LikeAction"/>
<meta itemprop="userInteractionCount" content="3" />
<span class='post-likes'>3 Likes</span>
<meta itemprop="userInteractionCount" content="4" />
<span class='post-likes'>4 Likes</span>
</div>

<div itemprop="interactionStatistic" itemscope itemtype="http:https://schema.org/InteractionCounter">
Expand Down
7 changes: 3 additions & 4 deletions scraper-post-process.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import { readJSON, writeJSON } from 'https://deno.land/x/flat/mod.ts'
import { cheerio } from "https://deno.land/x/[email protected]/mod.ts";

interface Post {
Expand Down Expand Up @@ -37,7 +36,7 @@ await posts.each(function(index, data) {
);
const dataviewCodes = rawData.filter(post => post.content?.includes('```dataview'));
for (const codePosts of dataviewCodes){
const fileName =`${codePosts.creator?.name || ''}_${codePosts.datePublished?.getTime()}.json`
await writeJSON(`./scripts/${fileName}`, codePosts);
const content = `---\ntitle:\nauthor:${codePosts?.creator?.name} (${codePosts?.creator?.url})\npublication date: ${codePosts.datePublished}\n---\n\n ${codePosts.content}`
const fileName =`${codePosts.creator?.name || ''}_${codePosts.datePublished?.getTime()}.md`
await Deno.writeTextFile(`./scripts/${fileName}`, content);
}
console.log(JSON.stringify(dataviewCodes));// Pluck a specific key off

0 comments on commit 7555fce

Please sign in to comment.