Skip to content

Commit

Permalink
Docs/code snippet tag2 (dagster-io#23117)
Browse files Browse the repository at this point in the history
## Summary & Motivation

After ruling out the "Client fetches code snippets from github"
approach, I ended up reimplementing it to be handled on the server. I've
also updated the github docs build action to watch the examples folder
as a location for updates and copy the content of the examples into the
next site when it is time to update.

### What was added
- A new next API route (`/api/code-snippet`) to handle fetching and
processing code snippets.
- A React component (CodeSnippet) to render these snippets in the
documentation.
- Markdoc tag declaration for easy use of the CodeSnippet component in
markdown files.
- Environment-aware file path handling for both local development and
production.
- Updates to the GitHub Actions workflow to copy snippet files to the
correct location during deployment.
- Fixes to an animation bug that emerged during the work.



## How I Tested These Changes

### Local Development:
- I tested the API route with various parameter combinations to ensure
correct snippet extraction.
- Verified that the React component correctly renders snippets fetched
from the API.
- Confirmed that the environment-aware path handling works correctly in
the local development environment.


### Markdoc Integration:
- Created sample Markdoc files with various CodeSnippet tag usages to
ensure proper rendering.
- Tested different parameter combinations within Markdoc tags.


### Production Simulation:
- Simulated a production environment locally to verify that file paths
are correctly resolved.
- Tested the GitHub Actions workflow in a feature branch to ensure
snippet files are correctly copied during deployment.


### Edge Cases
- Tested error handling for non-existent files, invalid parameters, and
network issues.
- Verified behavior with empty snippets, very large snippets, and
snippets with special characters.


### Performance
- Conducted basic performance tests to ensure that snippet loading
doesn't significantly impact page load times.
  • Loading branch information
nikomancy committed Jul 24, 2024
1 parent bd5f062 commit dc5a394
Show file tree
Hide file tree
Showing 7 changed files with 379 additions and 47 deletions.
13 changes: 13 additions & 0 deletions .github/workflows/build-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,13 @@ on:
- docs-prod # prod
paths:
- docs/**
- examples/doc_snippets/**
- CHANGES.md
- .github/workflows/build-docs.yml
pull_request:
paths:
- docs/**
- examples/doc_snippets/**
- CHANGES.md
- .github/workflows/build-docs.yml
concurrency:
Expand Down Expand Up @@ -67,6 +69,11 @@ jobs:
env:
GITHUB_HEAD_SHA: ${{ github.event.pull_request.head.sha }}

- name: Copy doc snippets to public directory
run: |
mkdir -p docs/next/public/docs_snippets
cp -R examples/docs_snippets/docs_snippets docs/next/public/docs_snippets/
- name: Publish Preview to Vercel
uses: amondnet/vercel-action@v25
if: |
Expand All @@ -86,6 +93,12 @@ jobs:
if: github.event_name == 'push' && github.ref == 'refs/heads/docs-prod'
uses: actions/checkout@v4

- name: Copy doc snippets to public directory (Production)
if: github.event_name == 'push' && github.ref == 'refs/heads/docs-prod'
run: |
mkdir -p docs/next/public/docs_snippets
cp -R examples/docs_snippets/docs_snippets docs/next/public/docs_snippets/
- name: Publish to Vercel Production
uses: amondnet/vercel-action@v25
if: github.event_name == 'push' && github.ref == 'refs/heads/docs-prod'
Expand Down
71 changes: 71 additions & 0 deletions docs/markdoc-component-documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ title: Authoring Component Examples
description: This page shows off what each of the components we're using in our docs looks like.
---



This page contains examples of all of the native Markdoc node and the custom markdoc tags that we are using in our docs.

To see the rendered version of these tags, move this file into the `next/pages/getting-started` directory, run the local server for the site, and navigate to the `getting-started/markdoc-component-documentation` page in your browser.
Expand Down Expand Up @@ -360,3 +362,72 @@ They work like this:
{% articleListItem title="Asset checks" href="/concepts/assets/asset-checks" /%}
{% articleListItem title="External assets (Experimental)" href="/concepts/assets/external-assets" /%}
{% /articleList %}

## CodeSnippets : Block

The `CodeSnippet` component allows you to easily include code snippets from your project files into your documentation. Here are various ways to use it, along with examples:

### Basic File Retrieval
This example shows how to include an entire file as a code snippet. It's useful when you want to showcase a complete file without any modifications.

`{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" /%}`

{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" /%}

### Specific Line Range
You can specify exact lines to include from a file. This is helpful when you want to focus on a particular section of code without showing the entire file.

`{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" lines="5-15" /%}`

{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" lines="5-15" /%}

### Multiple Line Ranges
For more complex scenarios, you can include multiple, non-contiguous line ranges. This allows you to showcase different parts of a file while skipping irrelevant sections.

`{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" lines="1-5,10-15" /%}`

{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" lines="1-5,10-15" /%}

### Start After a Specific String
This option lets you start the snippet after a specific string in the file. It's useful for beginning your snippet at a particular point, like after a comment or function definition.

`{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" startafter="# start_example" /%}`

{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" startafter="# start_example" /%}

### End Before a Specific String
Similar to `startafter`, this option lets you end the snippet before a specific string. It's helpful for showing code up to a certain point.

`{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" endbefore="# end_example" /%}`

{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" endbefore="# end_example" /%}

### Combine Start and End
You can use both `startafter` and `endbefore` to extract a specific section of code between two markers. This is great for showcasing a particular function or code block.

`{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" startafter="# start_example" endbefore="# end_example" /%}`

{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" startafter="# start_example" endbefore="# end_example" /%}

### Dedenting
The `dedent` option allows you to remove a specified number of leading spaces from each line. This is useful for adjusting the indentation of your snippet to match your documentation's formatting.

`{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" dedent=4 /%}`

{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" dedent=4 /%}

### Disable Trimming
By default, the component trims whitespace from the beginning and end of the snippet. You can disable this behavior if you need to preserve exact whitespace.

`{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" trim=false /%}`

{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" trim=false /%}

### Combining Multiple Parameters
You can combine multiple parameters for fine-grained control over your snippets. This example shows how to select specific lines, start after a marker, dedent, and trim the result.

`{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" lines="5-15" startafter="# start_example" dedent=4 trim=true /%}`

{% codeSnippet file="concepts/assets/asset_group_argument.py" lang="python" lines="5-15" startafter="# start_example" dedent=4 trim=true /%}

By using these options, you can flexibly include and format code snippets to best suit your needs.
58 changes: 58 additions & 0 deletions docs/next/components/markdoc/CodeSnippet.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
import React, { useState, useEffect } from 'react';
import { Fence } from './FencedCodeBlock';

interface CodeSnippetProps {
file: string;
lang: string;
lines?: string;
startafter?: string;
endbefore?: string;
dedent?: number;
trim?: boolean;
}

const API_BASE_URL = process.env.NEXT_PUBLIC_API_BASE_URL || '';

const fetchSnippet = async (params: CodeSnippetProps) => {
const queryParams = new URLSearchParams({
file: params.file,
...(params.lines && { lines: params.lines }),
...(params.startafter && { startafter: params.startafter }),
...(params.endbefore && { endbefore: params.endbefore }),
...(params.dedent && { dedent: params.dedent.toString() }),
...(params.trim !== undefined && { trim: params.trim.toString() }),
});

const response = await fetch(`${API_BASE_URL}/api/code-snippet?${queryParams}`);
if (!response.ok) {
throw new Error('Failed to fetch snippet');
}
return response.text();
};

export const CodeSnippet: React.FC<CodeSnippetProps> = (props) => {
const [snippet, setSnippet] = useState<string>('');
const [isLoading, setIsLoading] = useState<boolean>(true);
const [error, setError] = useState<string | null>(null);

useEffect(() => {
const fetchData = async () => {
try {
setIsLoading(true);
const data = await fetchSnippet(props);
setSnippet(data);
} catch (err) {
setError(err.message);
} finally {
setIsLoading(false);
}
};

fetchData();
}, [props]);

if (isLoading) return <div>Loading snippet...</div>;
if (error) return <div>Error: {error}</div>;

return <Fence data-language={props.lang}>{snippet}</Fence>;
};
82 changes: 36 additions & 46 deletions docs/next/components/markdoc/FencedCodeBlock.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ export const Fence = (props) => {

React.useEffect(() => {
Prism.highlightAll();
}, []);
}, [text]);

const copyToClipboard = React.useCallback(() => {
if (typeof text === 'string') {
Expand All @@ -27,53 +27,43 @@ export const Fence = (props) => {
}, [copy, text]);

return (
<div className="codeBlock relative" aria-live="polite" style={{display: 'flex'}}>
<pre className="line-numbers w-full">
<div className="codeBlock relative" aria-live="polite">
<pre className="line-numbers">
<code className={`language-${language}`}>{text}</code>
</pre>
<Transition
show={!copied}
appear={true}
enter="transition ease-out duration-150 transform"
enterFrom="opacity-0 scale-95"
enterTo="opacity-100 scale-100"
leave="transition ease-in duration-150 transform"
leaveFrom="opacity-100 scale-100"
leaveTo="opacity-0 scale-95"
>
<div className="absolute top-2 right-1 mt-2 mr-2">
<svg
className="h-5 w-5 text-gray-400 cursor-pointer hover:text-gray-300"
fill="none"
viewBox="0 0 24 24"
stroke="currentColor"
onClick={copyToClipboard}
>
<path
strokeLinecap="round"
strokeLinejoin="round"
strokeWidth="2"
d="M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2"
/>
</svg>
</div>
</Transition>
<Transition
show={copied}
appear={true}
enter="transition ease-out duration-150 transform"
enterFrom="opacity-0 scale-95"
enterTo="opacity-500 scale-100"
leave="transition ease-in duration-200 transform"
leaveFrom="opacity-100 scale-100"
leaveTo="opacity-0 scale-95"
>
<div className="absolute top-2 right-1 mt-1 mr-2">
<span className="select-none inline-flex items-center px-2 rounded text-xs font-medium leading-4 bg-gray-900 text-gray-400">
Copied
</span>
</div>
</Transition>
<div className="absolute top-2 right-2">
<Transition
show={true}
appear={true}
enter="transition-opacity duration-150"
enterFrom="opacity-0"
enterTo="opacity-100"
leave="transition-opacity duration-150"
leaveFrom="opacity-100"
leaveTo="opacity-0"
>
{copied ? (
<span className="select-none inline-flex items-center px-2 rounded text-xs font-medium leading-4 bg-gray-900 text-gray-400">
Copied
</span>
) : (
<svg
className="h-5 w-5 text-gray-400 cursor-pointer hover:text-gray-300"
fill="none"
viewBox="0 0 24 24"
stroke="currentColor"
onClick={copyToClipboard}
>
<path
strokeLinecap="round"
strokeLinejoin="round"
strokeWidth="2"
d="M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2"
/>
</svg>
)}
</Transition>
</div>
</div>
);
};
15 changes: 15 additions & 0 deletions docs/next/markdoc/tags.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import {Button, ButtonContainer} from '../components/markdoc/Button';
import {Note, Warning} from '../components/markdoc/Callouts';
import {Check, Cross} from '../components/markdoc/CheckCross';
import {CodeReferenceLink} from '../components/markdoc/CodeReferenceLink';
import {CodeSnippet} from '../components/markdoc/CodeSnippet';
import {MyImage} from '../components/markdoc/Image';
import {ReferenceTable, ReferenceTableItem} from '../components/markdoc/ReferenceTable';

Expand Down Expand Up @@ -132,3 +133,17 @@ export const articleListItem = {
},
},
};

export const codeSnippet = {
render: CodeSnippet,
selfClosing: true,
attributes: {
file: {type: String, required: true},
lang: {type: String, required: true},
lines: {type: String},
startafter: {type: String},
endbefore: {type: String},
dedent: {type: Number},
trim: {type: Boolean, default: true},
},
};
2 changes: 1 addition & 1 deletion docs/next/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "PORT=3001 next-remote-watch ../content",
"dev": "NEXT_PUBLIC_ENV=development PORT=3001 next-remote-watch ../content",
"build": "next build",
"build-master": "VERSIONING_DISABLED=true next build",
"start": "next start",
Expand Down
Loading

0 comments on commit dc5a394

Please sign in to comment.