Feat: @astrojs/rss package! (withastro#3271)

* feat: introduce @astrojs/rss package! * feat: add config "site" to env variable * docs: add @astrojs/rss readme * chore: changeset * fix: testing script * deps: add mocha, chai, chai-promises * tests: add rss test! * feat: add canonicalUrl arg * chore: remove console.log * fix: remove null check on env (breaks build) * docs: stray ` * chore: update error message to doc link * chore: remove getStylesheet * docs: update stylesheet reference
SiriousHunter · May 3, 2022 · fbfb619 · fbfb619
1 parent e2a037b
commit fbfb619
Show file tree

Hide file tree

Showing 9 changed files with 538 additions and 0 deletions.
diff --git a/.changeset/beige-gifts-yawn.md b/.changeset/beige-gifts-yawn.md
@@ -0,0 +1,6 @@
+---
+'astro': patch
+'@astrojs/rss': patch
+---
+
+Introduce new @astrojs/rss package for RSS feed generation! This also adds a new global env variable for your project's configured "site": import.meta.env.SITE. This is consumed by the RSS feed helper to generate the correct canonical URL.
diff --git a/packages/astro-rss/README.md b/packages/astro-rss/README.md
@@ -0,0 +1,147 @@
+# @astrojs/rss 📖
+
+This package brings fast RSS feed generation to blogs and other content sites built with [Astro](https://astro.build/). For more information about RSS feeds in general, see [aboutfeeds.com](https://aboutfeeds.com/).
+
+## Installation
+
+Install the `@astrojs/rss` package into any Astro project using your preferred package manager:
+
+```bash
+# npm
+npm i @astrojs/rss
+# yarn
+yarn add @astrojs/rss
+# pnpm
+pnpm i @astrojs/rss
+```
+
+## Example usage
+
+The `@astrojs/rss` package provides helpers for generating RSS feeds within [Astro endpoints][astro-endpoints]. This unlocks both static builds _and_ on-demand generation when using an [SSR adapter](https://docs.astro.build/en/guides/server-side-rendering/#enabling-ssr-in-your-project).
+
+For instance, say you need to generate an RSS feed for all posts under `src/pages/blog/`. Start by [adding a `site` to your project's `astro.config` for link generation](https://docs.astro.build/en/reference/configuration-reference/#site). Then, create an `rss.xml.js` file under your project's `src/pages/` directory, and use [Vite's `import.meta.glob` helper](https://vitejs.dev/guide/features.html#glob-import) like so:
+
+```js
+// src/pages/rss.xml.js
+import rss from '@astrojs/rss';
+
+export const get = () => rss({
+ title: 'Buzz’s Blog',
+ description: 'A humble Astronaut’s guide to the stars',
+ items: import.meta.glob('./blog/**/*.md'),
+ });
+```
+
+Read **[Astro's RSS docs][astro-rss]** for full usage examples.
+
+## `rss()` configuration options
+
+The `rss` default export offers a number of configuration options. Here's a quick reference:
+
+```js
+rss({
+ // `<title>` field in output xml
+ title: 'Buzz’s Blog',
+ // `<description>` field in output xml
+ description: 'A humble Astronaut’s guide to the stars',
+ // list of `<item>`s in output xml
+ items: import.meta.glob('./**/*.md'),
+ // (optional) absolute path to XSL stylesheet in your project
+ stylesheet: '/rss-styles.xsl',
+ // (optional) inject custom xml
+ customData: '<language>en-us</language>',
+ // (optional) add arbitrary metadata to opening <rss> tag
+ xmlns: { h: 'http:https://www.w3.org/TR/html4/' },
+ // (optional) provide a canonical URL
+ // defaults to the "site" configured in your project's astro.config
+ canonicalUrl: 'https://stargazers.club',
+});
+```
+
+### title
+
+Type: `string (required)`
+
+The `<title>` attribute of your RSS feed's output xml.
+
+### description
+
+Type: `string (required)`
+
+The `<description>` attribute of your RSS feed's output xml.
+
+### items
+
+Type: `RSSFeedItem[] | GlobResult (required)`
+
+Either a list of formatted RSS feed items or the result of [Vite's `import.meta.glob` helper](https://vitejs.dev/guide/features.html#glob-import). See [Astro's RSS items documentation](https://docs.astro.build/en/guides/rss/#generating-items) for usage examples to choose the best option for you.
+
+When providing a formatted RSS item list, see the `RSSFeedItem` type reference below:
+
+```ts
+type RSSFeedItem = {
+ /** Link to item */
+ link: string;
+ /** Title of item */
+ title: string;
+ /** Publication date of item */
+ pubDate: Date;
+ /** Item description */
+ description?: string;
+ /** Append some other XML-valid data to this item */
+ customData?: string;
+};
+```
+
+### stylesheet
+
+Type: `string (optional)`
+
+An absolute path to an XSL stylesheet in your project. If you don’t have an RSS stylesheet in mind, we recommend the [Pretty Feed v3 default stylesheet](https://github.com/genmon/aboutfeeds/blob/main/tools/pretty-feed-v3.xsl), which you can download from GitHub and save into your project's `public/` directory.
+
+### customData
+
+Type: `string (optional)`
+
+A string of valid XML to be injected between your feed's `<description>` and `<item>` tags. This is commonly used to set a language for your feed:
+
+```js
+import rss from '@astrojs/rss';
+
+export const get = () => rss({
+ ...
+ customData: '<language>en-us</language>',
+ });
+```
+
+### xmlns
+
+Type: `Record<string, string> (optional)`
+
+An object mapping a set of `xmlns` suffixes to strings of metadata on the opening `<rss>` tag.
+
+For example, this object:
+
+```js
+rss({
+ ...
+ xmlns: { h: 'http:https://www.w3.org/TR/html4/' },
+})
+```
+
+Will inject the following XML:
+
+```xml
+<rss xmlns:h="http:https://www.w3.org/TR/html4/"...
+```
+
+### canonicalUrl
+
+Type: `string (optional)`
+
+The base URL to use when generating RSS item links. This defaults to the [`site` configured in your project's `astro.config`](https://docs.astro.build/en/reference/configuration-reference/#site). We recommend using `site` instead of `canonicalUrl`, though we provide this option if an override is necessary.
+
+For more on building with Astro, [visit the Astro docs][astro-rss].
+
+[astro-rss]: https://docs.astro.build/en/guides/rss/#using-astrojsrss-recommended
+[astro-endpoints]: https://docs.astro.build/en/core-concepts/astro-pages/#non-html-pages
diff --git a/packages/astro-rss/package.json b/packages/astro-rss/package.json
@@ -0,0 +1,39 @@
+{
+ "name": "@astrojs/rss",
+ "description": "Add RSS feeds to your Astro projects",
+ "version": "0.1.0",
+ "type": "module",
+ "types": "./dist/index.d.ts",
+ "author": "withastro",
+ "license": "MIT",
+ "repository": {
+ "type": "git",
+ "url": "https://github.com/withastro/astro.git",
+ "directory": "packages/astro-rss"
+ },
+ "bugs": "https://github.com/withastro/astro/issues",
+ "homepage": "https://astro.build",
+ "exports": {
+ ".": "./dist/index.js",
+ "./package.json": "./package.json"
+ },
+ "scripts": {
+ "build": "astro-scripts build \"src/**/*.ts\" && tsc",
+ "build:ci": "astro-scripts build \"src/**/*.ts\"",
+ "dev": "astro-scripts dev \"src/**/*.ts\"",
+ "test": "mocha --exit --timeout 20000"
+ },
+ "devDependencies": {
+ "@types/chai": "^4.3.1",
+ "@types/chai-as-promised": "^7.1.5",
+ "@types/mocha": "^9.1.1",
+ "astro": "workspace:*",
+ "astro-scripts": "workspace:*",
+ "chai": "^4.3.6",
+ "chai-as-promised": "^7.1.1",
+ "mocha": "^9.2.2"
+ },
+ "dependencies": {
+ "fast-xml-parser": "^4.0.7"
+ }
+}
diff --git a/packages/astro-rss/src/index.ts b/packages/astro-rss/src/index.ts
@@ -0,0 +1,154 @@
+import { XMLValidator } from 'fast-xml-parser';
+import { createCanonicalURL, isValidURL } from './util.js';
+
+type GlobResult = Record<string, () => Promise<{ [key: string]: any }>>;
+
+type RSSOptions = {
+ /** (required) Title of the RSS Feed */
+ title: string;
+ /** (required) Description of the RSS Feed */
+ description: string;
+ /**
+ * List of RSS feed items to render. Accepts either:
+ * a) list of RSSFeedItems
+ * b) import.meta.glob result. You can only glob ".md" files within src/pages/ when using this method!
+ */
+ items: RSSFeedItem[] | GlobResult;
+ /** Specify arbitrary metadata on opening <xml> tag */
+ xmlns?: Record<string, string>;
+ /**
+ * Specifies a local custom XSL stylesheet. Ex. '/public/custom-feed.xsl'
+ */
+ stylesheet?: string | boolean;
+ /** Specify custom data in opening of file */
+ customData?: string;
+ /**
+ * Specify the base URL to use for RSS feed links.
+ * Defaults to "site" in your project's astro.config
+ */
+ canonicalUrl?: string;
+};
+
+type RSSFeedItem = {
+ /** Link to item */
+ link: string;
+ /** Title of item */
+ title: string;
+ /** Publication date of item */
+ pubDate: Date;
+ /** Item description */
+ description?: string;
+ /** Append some other XML-valid data to this item */
+ customData?: string;
+};
+
+type GenerateRSSArgs = {
+ site: string;
+ rssOptions: RSSOptions;
+ items: RSSFeedItem[];
+};
+
+function isGlobResult(items: RSSOptions['items']): items is GlobResult {
+ return typeof items === 'object' && !items.length;
+}
+
+function mapGlobResult(items: GlobResult): Promise<RSSFeedItem[]> {
+ return Promise.all(
+ Object.values(items).map(async (getInfo) => {
+ const { url, frontmatter } = await getInfo();
+ if (!Boolean(url)) {
+ throw new Error(
+ `[RSS] When passing an import.meta.glob result directly, you can only glob ".md" files within /pages! Consider mapping the result to an array of RSSFeedItems. See the RSS docs for usage examples: https://docs.astro.build/en/guides/rss/#2-list-of-rss-feed-objects`
+ );
+ }
+ if (!Boolean(frontmatter.title) || !Boolean(frontmatter.pubDate)) {
+ throw new Error(`[RSS] "${url}" is missing a "title" and/or "pubDate" in its frontmatter.`);
+ }
+ return {
+ link: url,
+ title: frontmatter.title,
+ pubDate: frontmatter.pubDate,
+ description: frontmatter.description,
+ customData: frontmatter.customData,
+ };
+ })
+ );
+}
+
+export default async function getRSS(rssOptions: RSSOptions) {
+ const site = rssOptions.canonicalUrl ?? (import.meta as any).env.SITE;
+ if (!site) {
+ throw new Error(
+ `RSS requires a canonical URL. Either add a "site" to your project's astro.config, or supply the canonicalUrl argument.`
+ );
+ }
+ let { items } = rssOptions;
+ if (isGlobResult(items)) {
+ items = await mapGlobResult(items);
+ }
+ return {
+ body: await generateRSS({
+ site,
+ rssOptions,
+ items,
+ }),
+ };
+}
+
+/** Generate RSS 2.0 feed */
+export async function generateRSS({ site, rssOptions, items }: GenerateRSSArgs): Promise<string> {
+ let xml = `<?xml version="1.0" encoding="UTF-8"?>`;
+ if (typeof rssOptions.stylesheet === 'string') {
+ xml += `<?xml-stylesheet href="${rssOptions.stylesheet}" type="text/xsl"?>`;
+ }
+ xml += `<rss version="2.0"`;
+
+ // xmlns
+ if (rssOptions.xmlns) {
+ for (const [k, v] of Object.entries(rssOptions.xmlns)) {
+ xml += ` xmlns:${k}="${v}"`;
+ }
+ }
+ xml += `>`;
+ xml += `<channel>`;
+
+ // title, description, customData
+ xml += `<title><![CDATA[${rssOptions.title}]]></title>`;
+ xml += `<description><![CDATA[${rssOptions.description}]]></description>`;
+ xml += `<link>${createCanonicalURL(site).href}</link>`;
+ if (typeof rssOptions.customData === 'string') xml += rssOptions.customData;
+ // items
+ for (const result of items) {
+ xml += `<item>`;
+ xml += `<title><![CDATA[${result.title}]]></title>`;
+ // If the item's link is already a valid URL, don't mess with it.
+ const itemLink = isValidURL(result.link)
+ ? result.link
+ : createCanonicalURL(result.link, site).href;
+ xml += `<link>${itemLink}</link>`;
+ xml += `<guid>${itemLink}</guid>`;
+ if (result.description) xml += `<description><![CDATA[${result.description}]]></description>`;
+ if (result.pubDate) {
+ // note: this should be a Date, but if user provided a string or number, we can work with that, too.
+ if (typeof result.pubDate === 'number' || typeof result.pubDate === 'string') {
+ result.pubDate = new Date(result.pubDate);
+ } else if (result.pubDate instanceof Date === false) {
+ throw new Error('[${filename}] rss.item().pubDate must be a Date');
+ }
+ xml += `<pubDate>${result.pubDate.toUTCString()}</pubDate>`;
+ }
+ if (typeof result.customData === 'string') xml += result.customData;
+ xml += `</item>`;
+ }
+
+ xml += `</channel></rss>`;
+
+ // validate user’s inputs to see if it’s valid XML
+ const isValid = XMLValidator.validate(xml);
+ if (isValid !== true) {
+ // If valid XML, isValid will be `true`. Otherwise, this will be an error object. Throw.
+ throw new Error(isValid as any);
+ }
+
+ return xml;
+}
diff --git a/packages/astro-rss/src/util.ts b/packages/astro-rss/src/util.ts
@@ -0,0 +1,19 @@
+import npath from 'path-browserify';
+
+/** Normalize URL to its canonical form */
+export function createCanonicalURL(url: string, base?: string): URL {
+ let pathname = url.replace(/\/index.html$/, ''); // index.html is not canonical
+ pathname = pathname.replace(/\/1\/?$/, ''); // neither is a trailing /1/ (impl. detail of collections)
+ if (!npath.extname(pathname)) pathname = pathname.replace(/(\/+)?$/, '/'); // add trailing slash if there’s no extension
+ pathname = pathname.replace(/\/+/g, '/'); // remove duplicate slashes (URL() won’t)
+ return new URL(pathname, base);
+}
+
+/** Check if a URL is already valid */
+export function isValidURL(url: string): boolean {
+ try {
+ new URL(url);
+ return true;
+ } catch (e) {}
+ return false;
+}