
&nbsp;&nbsp;&nbsp; Ask yourself: Does your favorite social media company need to know your birth date?
No! They will argue that they need to verify your age, such as 13, to which I will
respond by informing them of a technology called "zero-knowledge proofs" that lets
their users prove just what the companies _need_ to know (being above the age of 13) without revealing anything else (such as actual birthdate, name, or address).
The internet has undergone a series of changes. First, it was a one-way street with
businesses presenting information to consumers. Then it became interactive as users
signed up for services like Facebook and stored personal information (such as email,
passwords, name, and address) in their data centers. Once data collection reached
large volumes, became more complex, and was shared/sold with growing velocity, it
was described as "big data." In came the "data wranglers" to wrangle the data with
machine learning and extract profits from users. Now we are left with numerous business
models that would not exist if it were not so easy to metaphorically walk around
with a clipboard and follow millions of people 24/7, jotting down their every thought,
action, and revealed interests. If you want to be in control of your data and privacy,
then pay attention to zero-knowledge proofs, which academics from top universities
like UC Berkeley hope will bring a new privacy layer to the internet that walls off
companies tracking you and their disingenuous business models.

### What are zero-knowledge proofs?

&nbsp;&nbsp;&nbsp; Zero-knowledge proofs let you make assertions and prove them without revealing anything but the claim itself. It is as if every question was a yes/no or multiple choice, and you just had to bubble in your answer without showing your work. Where can this be used? Everywhere, maybe? Every website likes to verify your identity by having you type in a username and password, but imagine the login screen being yes/no: are you Alice, yes or no? As Howard Wu writes in his article,

> "With zero-knowledge proofs, however, users can now hash their password on-device, that is, without ever having to send their password to any web service. Imagine never having your password compromised again because of others’ errors, bad actors, or reasons beyond your control."

Beyond passwords, many people also care about keeping their finances and health data private. Today, when you rent or buy a home, real estate agents ask for complete bank account info and recent paychecks to check if you meet a minimum qualifying income. Later, they can use this data to send targeted ads whether or not you made a transaction with them. Alternatively, zero-knowledge proofs would just let you check the box: do you meet this minimum income requirement, yes/no? While they seem magical, we’ll now begin to better understand how they work.

### How zero-knowledge proofs work

&nbsp;&nbsp;&nbsp; Zero-Knowledge proofs were first introduced in 1985 by a UC Berkeley researcher and Turing award winner, Shafi Goldwasser, and her colleagues in their paper “The Knowledge Complexity of Interactive Systems.” One of the earliest descriptions of how they work comes from a real-world situation; Carlos is a non-colorblind person who needs to prove that he’s not colorblind to a friend. One way to do this is for the friend to get medical records from Carlos’s doctor, however, Carlos doesn’t want to share his sensitive health records. Instead, the friend holds two markers of different colors behind their back. The friend then reveals one marker to Carlos, puts it behind their back again, then switches or doesn’t switch the markers before revealing the marker again and asking Carlos if it’s the same marker. They’ll repeat this until the friend is convinced that Carlos isn’t colorblind. This type of proof is called an “interactive proof,” one of the first iterations of a proof-generating protocol.

### Digital interactive zero-knowledge proofs

&nbsp;&nbsp;&nbsp; In the digital world, zero-knowledge proofs work pretty much the same way but with a bit more math and cryptography backing them. Suppose there’s an online Sudoku competition happening and Alice finds a solution that will lead to a prize and she wants to prove it to judges and other participants without revealing it. Sudoku is a game where there’s a gridded board with some cells filled in and some cells aren't, and the challenge is to fill in all the grid cells without violating the game's rules. The interactive zero-knowledge proof is a multi-step process:

1. Alice creates a random “permutation” of her solution, each value 0-9 is mapped to a new unique value. She can’t send this permuted board over because then people might be able to derive all the original values.
2. Alice then assigns each cell a random “nonce” which is a few random letters.
3. Each cell’s nonce and permuted value get “hashed” (combined) into a new random string of letters that could only be produced by the permuted value and its nonce.
4. Finally, Alice sends the entire board of hashed values to the judges. The judges can’t derive the permuted values (and then the entire solution) because hash functions are one-way.
5. Instead, the judges can only ask Alice one question: reveal the permuted values and their nonces of some row, column, or subgrid, and the permuted values of a few other cells that were present on the unsolved sudoku puzzle. If Alice’s solution is _likely_ to be correct, then:
   - No row, column, or subgrid can contain duplicate values.
   - The nonce for each cell and the permuted value should hash to the same hashes on the hashed board they received.
   - If the original sudoku board had an, eg. “1”, value in the row that the judges selected and a “1” value in the random cell they selected, then the permuted values should both be the same; this would mean that Alice’s permuted board is consistent.
6. If step 5 went well, then Alice and the judges can repeat all the previous 5 steps until the judges are convinced that she has a solution and can award her the prize.

A pictorial writeup for this was written by Manish Goregaokar, who also created an interactive game and demonstration for it that readers can [check out](https://manishearth.github.io/sudoku-zkp/zkp.html).

### Non-interactive zero-knowledge proofs

&nbsp;&nbsp;&nbsp; This brings us to today, instead of interactive zero-knowledge proofs which aren’t practical or efficient in the real world, there are now \_non-interactive \_zero-knowledge proofs. As the [Ethereum ](https://ethereum.org/en/zero-knowledge-proofs)Foundation explains,

> “While revolutionary, interactive proving had limited usefulness since it required the two parties to be available and interact repeatedly. Even if a verifier was convinced of a prover’s honesty, the proof would be unavailable for independent verification (computing a new proof required a new set of messages between the prover and verifier).

> To solve this problem, Manuel Blum, Paul Feldman, and Silvio Micali suggested the first non-interactive zero-knowledge proofs where the prover and verifier have a shared key. This allows the prover to demonstrate their knowledge of some information (i.e., witness) without providing the information itself.”

Illustrating this non-interactive zero-knowledge protocol, we can again use the Sudoku example. The protocol, a third-party, would provide Alice and the competition judges with their own “key,” a string of random letters and numbers. Alice uses her key to hash /mask her solution and she sends it to the judges in one message. The judges then use their special verification key to verify it. While it sounds simple, there are years of academic research in math and cryptography that led to it. However, there are still hurdles to overcome as zero-knowledge proofs can’t be trivially applied to any ordinary problem. Information that would usually be shared to prove a statement or answer yes/no questions needs to be translated by cryptographers into code and math that lets users encrypt it on their end and have it verified on the verifier's end. At the moment, this requires nearly unique zero-knowledge protocols for every unique problem, which explains why there are so many zero-knowledge protocols and research papers. Working to address this is the [ZKProof Standards](https://zkproof.org/) institute.

### Critics

&nbsp;&nbsp;&nbsp; Not all are so bullish on zero-knowledge-proof technology and encryption that protects user privacy. As New York Times journalist Ali Asaei writes about the messaging app Signal, an app that uses end-to-end encryption and had its founder Jack Dorsey state that they’d never and couldn’t reveal user messages, even if law enforcement asked,

> “I [Ali] am drawing attention to Signal, but there’s a bigger issue here: Small groups of technologists are developing and deploying applications of their technologies for explicitly ideological reasons, with those ideologies baked into the technologies. To use those technologies is to use a tool that comes with an ethical or political bent. (…) Whether law enforcement should tap our phones on the condition that a warrant is obtained is, at the very least, worthy of public discussion. Signal has unilaterally decided for us all.”

Asei’s argument is reasonable, zero-knowledge proofs may lead to a world where people have too much privacy and can conduct illegal activities without being caught. Therefore, like any new technology, we need to weigh the costs and benefits and preemptively mitigate any problems that we can foresee. Artificial intelligence has to do the same, so does online gambling, and so did cars and airplanes when they were first introduced. I’m bullish on zero-knowledge proofs because we’ve had years of regulations like GDPR and California’s Online Privacy Act that have tried to protect our online privacy but have failed. After all, they don’t address the root problem and don’t have the resources to verify every company is following the rules. Zero-knowledge proofs address the root problem: they put us in control and let us share the precise information we choose to share and nothing more.

#### References

> Asaei, Ali. “Opinion | Jack Dorsey and the Dangers of Privacy at All Costs - the New York Times.” _Web.archive.org_, 1 Jan. 2023, web.archive.org/web/20230101194846/www.nytimes.com/2022/12/28/opinion/jack-dorseys-twitter-signal-privacy.html. Accessed 21 Feb. 2023.

> BBVA. “Zero Knowledge Proof: How to Maintain Privacy in a Data-Based World.” _NEWS BBVA_, 11 Sept. 2019, www.bbva.com/en/zero-knowledge-proof-how-to-maintain-privacy-in-a-data-based-world/.

> “Homepage.” _ZKProof Standards_, zkproof.org/.

> “Interactive Sudoku Prover.” _Manishearth.github.io_, manishearth.github.io/sudoku-zkp/zkp.html. Accessed 21 Feb. 2023.

> “Privacy-Protecting Regulatory Solutions Using Zero-Knowledge Proofs: Full Paper.” _A16z Crypto_, 16 Nov. 2022, a16zcrypto.com/privacy-protecting-regulatory-solutions-using-zero-knowledge-proofs-full-paper/. Accessed 21 Feb. 2023.

> “Why Zero-Knowledge KYC Won’t Work.” _Cointelegraph_, cointelegraph.com/news/why-zero-knowledge-kyc-won-t-work. Accessed 21 Feb. 2023.

> Wu, Howard. “How the Coming Privacy Layer Will Fix the Broken Web.” _Future_, 15 June 2021, future.com/a-privacy-layer-for-the-web-can-change-everything/. Accessed 21 Feb. 2023.

> “Zero-Knowledge Proofs.” _Ethereum.org_, ethereum.org/en/zero-knowledge-proofs/.


Can Zero-Knowledge-Proofs Protect User Privacy?


Textual content is everywhere: websites, notes, books, emails, presentations, and developer documentation. It's just presented differently.

It all has a similar structure: Headers, sub-headers, paragraphs, **bolded** and _italicized_ _words_, lists, tables, images, and links. The tool you use just may be different.

Word, PDF, HTML, Apple Notes, why is it so hard to convert between them? Why can't it just be plain text?

[Markdown](https://www.markdownguide.org/) and [the unified collective](https://github.com/unifiedjs/unified) is the answer.

## What's Markdown?

Markdown is a universal language that lets you write plain text documents that contain everything that a Word document does and _more_.

Instead of using buttons on a property platform, or writing HTML code yourself to add headers and such, you just add a character or two, like a '#' to signify the text is a header. `# Header` `## Subheader`

You then take your plain text document anywhere that supports Markdown (a lot of places these days) and they can render it to look how you want (book, word document, website, etc).

![Markdown Flowchart](./markdown-flowchart.png)

## What's the Unified Collective?

Markdown language and syntax need to be transformed into your final desired format (HTML, PDF, etc), and even vice-versa (HTML to Markdown). The Unified Collective builds the tools to do this.

Transforming content has 3 main steps:

1.  **Parsing**: Content needs to be turned into a "syntax tree" as specified by [unist](https://github.com/syntax-tree/unist). Unist doesn't exactly specify what a syntax tree is for different content forms like Markdown/HTML input. It just defines a _universal_ abstract syntax tree and expects projects to extend unist for specific languages. Here are popular specifications for different languages:

    - [mdast](https://github.com/syntax-tree/mdast) - Specifies **M**ark**d**own **a**bstract **s**yntax **t**rees.
    - [hast](https://github.com/syntax-tree/hast) - Specifies HTML abstract syntax trees.
    - [xast](https://github.com/syntax-tree/xast) - Specifies XML abstract syntax trees.
    - [esast](https://github.com/syntax-tree/esast) - Specifies JavaScript abstract syntax trees.

    The projects above _still_ only specify **abstract** syntax trees. To get a **concrete** syntax tree there's another collection of projects that do the actual parsing:

    - [remark](https://github.com/remarkjs/remark) for Markdown
    - [rehype](https://github.com/rehypejs/rehype) for HTML
    - [retext](https://github.com/retextjs/retext) for natural language

2.  **Transformation**: The syntax trees (usually JSON format) can then be transformed into whatever you want by the ecosystem of **plugins** that exist for the concrete syntax trees.

    -     [remark plugins](https://github.com/remarkjs/remark/blob/main/doc/plugins.md)
    - [rehype plugins](https://github.com/rehypejs/rehype/blob/main/doc/plugins.md)
    - [retext plugins](https://github.com/retextjs/retext/blob/main/doc/plugins.md)

    For example, the [remark-pdf](https://github.com/inokawa/remark-pdf) plugin can turn markdown files into PDFs. If a plugin you want doesn't exist, you can [create your own](https://unifiedjs.com/learn/guide/create-a-plugin/). It's easy(ish) since you're working with a _concrete syntax tree_.

    While some plugins can completely change your concrete tree type, like the [remark-rehype](https://github.com/remarkjs/remark-rehype) plugin which turns **mdast** (markdown) trees into **hast** (HTML) trees, other plugins may just change the style by adding extra spaces, or checking for errors (linting) and outputting warning messages.

3.  **Stringify**: The transformed syntax tree is then stringified into its final form (HTML, markdown, prose/natural language). The output depends on the plugins you used.

### An example: transforming markdown to HTML

```javascript
import { unified } from "unified";
import remarkParse from "remark-parse";
import remarkRehype from "remark-rehype";
import rehypeSanitize from "rehype-sanitize";
import rehypeStringify from "rehype-stringify";

main();

async function main() {
  const file = await unified()
    .use(remarkParse) // specify the parser for the input file
    .use(remarkRehype) // turn the mdast tree into a hast tree
    .use(rehypeSanitize) // use a rehype plugin to ensure the HTML is safe
    .use(rehypeStringify) // stringify the hast tree into HTML
    .process("# Hello, Neptune!"); // do the 3 steps: parse, transform, and stringify using the tools above.

  console.log(String(file)); // --> <h1>Hello, Neptune!</h1>
}
```

## Markdown's limitations

Standard [CommonMark](https://commonmark.org/) or [GitHub Flavored Markdown](https://github.github.com/gfm/) is great for static text, links, and images, but it falls short on interactive content like charts and widgets.

JavaScript and ReactJS libraries are modern ways to create dynamic and interactive websites. But standard Markdown language doesn't support these.

[MDX](https://mdxjs.com/) extends the markdown syntax with [JSX](https://reactjs.org/docs/introducing-jsx.html) syntax. MDX files use the `.mdx` extension instead of the `.md` extension.

## How to process an MDX file

MDX files can be processed using a long list of remark plugins, starting with [remark-mdx](https://github.com/mdx-js/mdx/tree/main/packages/remark-mdx) which transforms the remark syntax tree (based on the mdast specification) into one that supports the MDX syntax.

You'd then need a lot of plugins afterward to parse the tree and stringify it into a [JSX Fragment](https://reactjs.org/docs/fragments.html).

Thankfully, the [mdx-js project](https://github.com/mdx-js/mdx) built a collection of packages to help you process an MDX file into React Fragments.

### MDX Processors

There are many MDX processers, but at the root of all of them is the [@mdx-js/mdx](https://mdxjs.com/packages/mdx/) compiler which uses a ton of remark plugins to turn MDX into a JSX Fragment. See the `mdx` [architecture](https://mdxjs.com/packages/mdx/#architecture) for how this works.

However, if you're using a bundler (see my [other note](/notes/modern-websites) on what that is) like webpack, Rollup, or esbuild, then you'll want to use something like the [@mdx-js/loader](https://mdxjs.com/packages/loader/) package. Adding this package into your webpack config will then enable you to import a `.mdx` file and get a React fragment returned.

```jsx
import Article from "./article.mdx";
export default function Page() {
  return <Article />;
}
```

But even more, if you're using a site builder like NextJS or Gatsby and you want to link directly to `.mdx` pages as you would for `.html` or `.jsx` pages, then you can use packages like [@next/mdx](https://github.com/vercel/next.js/tree/canary/packages/next-mdx) or [gatsby-plugin-mdx]() which further extend packages like [@mdx-js/loader](https://mdxjs.com/packages/loader/).

Assuming you have `pages/about.mdx`, in NextJS you can do something like:

```jsx
import Link from "next/link";
export default function HomePage() {
  return <Link href="/about">About Page</Link>;
}
```

### Processing MDX _On Demand_

Linking to MDX pages directly is great (as with `@next/mdx` or `gatsby-plugin-mdx`) but what if your `.mdx` file is in a remote repository, or if you don't know the page ahead of time because you're doing [dynamic routing](https://nextjs.org/docs/routing/dynamic-routes) (importing files during runtime)? This was my issue, I had a NextJS project structure laid out like this:

```bash
├── content
|  └── notes
|     ├── about
|     └──  index.mdx
|     └──  photo.png
|     └──  component.js
├── pages
|  ├── index.js
|  ├── notes
|  |  ├── index.js
|  |  └── [slug].js
|  ├── _app.js
|  └── _document.js
```

Inside `notes/[slug].js` I was doing dynamic routing to get the MDX note based on the URL route:

```javascript
import { getFileBySlug, getSlugs } from "@/lib/mdx";
import MdxLayout from "@/components/MdxLayout";

export async function getStaticPaths() {
  const slugs = getSlugs();
  return {
    paths: slugs.map((slug) => {
      params: {
        slug: slug;
      }
    }),
    fallback: false,
  };
}

export async function getStaticProps({ params }) {
  const slug = params.slug;
  let { content, metadata } = getFileBySlug(slug);
  return { props: { content, metadata } };
}

export default function SpecificNote({ content, metadata }) {
  const { title } = metadata;
  return (
    <MdxLayout>
      <h1>{title}</h1>
      // MDX content rendered somehow?????????????
    </MdxLayout>
  );
}
```

For rendering, I start with the `content` variable which is the raw text from the `.mdx` file. I need to convert this into a [`react fragment`](https://reactjs.org/docs/fragments.html) which contains HTML and JavaScript. Luckily, three popular projects can do this:

#### Comparing Three popular MDX on demand packages

- [`@mdx-js/mdx`](https://github.com/mdx-js/mdx/tree/main/packages/mdx)
- [`next-mdx-remote`](https://github.com/hashicorp/next-mdx-remote)
- [`mdx-bundler`](https://github.com/kentcdodds/mdx-bundler)
  <small>Note, the latter two extend the original `@mdx-js/mdx` package.</small>

So which do I choose? Here's an overview of what each does:

- `@mdx-js/mdx`
  - 1. [Parse](https://github.com/micromark/micromark) MDX (serialized markdown with [embedded JSX](https://github.com/micromark/micromark-extension-mdxjs), ESM, and expressions) to [mdast](https://github.com/syntax-tree/mdast) (markdown syntax tree)
  - 2.  [Transform](https://github.com/remarkjs/remark/blob/main/doc/plugins.md#list-of-plugins) through remark (markdown ecosystem)
  - 3.  [Transform](https://github.com/syntax-tree/mdast-util-to-hast) mdast to [hast](https://github.com/syntax-tree/hast) (HTML syntax tree)
  - 4.  [Transform](https://github.com/rehypejs/rehype/blob/main/doc/plugins.md#list-of-plugins) through rehype (HTML ecosystem)
  - 5.  [Transform](https://github.com/syntax-tree/hast-util-to-estree) hast to [esast](https://github.com/syntax-tree/esast) (JS syntax tree)
  - 6.  Do the [work](https://mdxjs.com/packages/mdx/#api) needed to get a [component](https://github.com/syntax-tree/hast-util-to-estree)
  - 7.  Transform through recma (JS ecosystem)
  - 8.  [Serialize](https://github.com/davidbonnet/astring) [esast](https://github.com/syntax-tree/esast) as JavaScript
  - The final output after [`compile`](https://mdxjs.com/packages/mdx/#compilefile-options) and [`run`](https://mdxjs.com/packages/mdx/#runfunctionbody-options) is a JSX fragment that we can use directly.
- `next-mdx-remote`
  - 1. [Serializes()](https://github.com/hashicorp/next-mdx-remote/blob/main/src/serialize.ts) the MDX by removing imports and exports from your MDX file and then calling `compile()` from `@mdx-js/mdx` which does steps 1-8 above and return a JSX Fragment as a string.
  - 2.  Renders the MDX with its [MDXRemote](https://github.com/hashicorp/next-mdx-remote/blob/main/src/index.tsx) component that takes a few options:
    - `source`: the serialized MDX string
    - `lazy`: boolean, if true it returns an empty div while the React component is created from the source string.
    - `components`: lets you pass components for a specific MDX file you're rendering
  - This is essentially abstracts a few steps you'd have to do and figure out if you had used the `@mdx-js/mdx` directly, not much more.
- `mdx-bundler`
  - 1. Creates a custom [esbuild](https://esbuild.github.io/) plugin using a variety of config options you give it.
  - 2. Then creates a [config](https://esbuild.github.io/api/) for esbuild using its plugin, the [`@mdx-js/esbuild`](https://github.com/mdx-js/mdx/tree/main/packages/esbuild) plugin, and the extra esbuild config options you pass to it.
  - 3.  Bundles your MDX file and returns a React component + frontmatter for you to use (b/c it includes the [`remark-frontmatter`](https://github.com/remarkjs/remark-frontmatter) plugin by default).
  - The key value prop of this package over the two above is that it lets you use import/export inside your MDX file. Importing/exporting is code-splitting, which requires bundlers like webpack or esbuild to find the files and piece them together.
  - The downside is that you're adding another bundler (`esbuild`) to your project which makes this package a bit heavy.

#### The package I chose

I settled for `mdx-bundler.` _Here's why_:

- To recap, I had a folder `content/notes/[slug]/` with files: `index.mdx`, `photo.png`, `ButtonComponent.js`. The `.mdx` file is written in plain text but it's supposed to be rendered with the pictures and the JavaScript it imports.
- I wanted to get the rendered package on the fly. Using the URL I'd match it to a "slug" and specific note.
- Although dynamically importing files with, `await import("content/notes/slug/index.mdx")` and using the `@next/mdx` webpack loader returns a React component, it has to be done fully on the **client-side** because NextJS only allows static and serializable content to be passed as props to pages. Thus, _I can't use the default MDX loaders._
- The two packages: `@mdx-js/mdx` and `next-mdx-remote` let you [compile](https://mdxjs.com/packages/mdx/#use) `.mdx` files into a JavaScript that builds and by default exports a React component when you run it. Thus, _while you can_ stringify this and pass it as a prop to pages, _you can't_ import pictures or components inside your `.mdx` files because these packages aren't bundlers so they can't resolve imports/exports.
- So, I'm left with `mdx-bundler`. Despite its large size, it does everything I need it to:
  - [x] I can store and use pictures and components right alongside the `.mdx` pages. My content can be anywhere.
  - [x] I can compile `.mdx` files during build time inside `getStaticProps()` (the most computationally expensive work) and then construct the React component quickly on the client-side during runtime.

#### The final result

Same file structure as described above, here's what I wrote:

```javascript
import MdxLayout from "@/components/MdxLayout";
import { formatDate } from "@/lib/utils";
import { useMemo } from "react";
import path from "path";
import { bundleMDX } from "mdx-bundler";
import { getMDXComponent } from "mdx-bundler/client";
import remarkGfm from "remark-gfm";
import remarkMdxImages from "remark-mdx-images";

// at build time, generate a map of all possible paths
export async function getStaticPaths() {
  const slugs = getSlugs();
  return {
    paths: slugs.map((slug) => {
      return { params: { slug: slug } };
    }),
    fallback: false,
  };
}

// at build time, map every path with ready-to-construct mdx component
export async function getStaticProps({ params }) {
  const slug = params.slug; // URL keyword (eg. "about")
  const slugDir = path.join(process.cwd(), "content", "notes", slug); // content/notes/about
  const publicDir = path.join(process.cwd(), "public", "static"); // public/static

  let loaderImages = {}; // {".png":"file", ".jpg":"file"}
  [".png", ".jpg"].forEach((ext) => (loaderImages[ext] = "file"));

  const { code, frontmatter } = await bundleMDX({
    file: `${slugDir}/index.mdx`, // entry point for esbuild to start resolving imports/exports
    cwd: `${slugDir}/`, // relative path that imports/exports will be using, like "./photo.png"
    bundleDirectory: `${publicDir}/notes/${slug}/`, // bundle images into this public folder
    bundlePath: `/static/notes/${slug}/`, // load all images with the prefix public subfolder prefix
    esbuildOptions: (options) => {
      options.loader = {
        ...options.loader, // custom mdx-bundler and @mdx/esbuild loader
        ...loaderImages, // our images are interpreted as files
        ".js": "jsx", //  .js files should be treated as .jsx files so they can export components
      };
      options.target = [
        // bundle to older js syntax so that older browsers support it
        "es2020",
        "chrome58",
        "firefox57",
        "node12",
        "safari11",
      ];
      options.define = {
        "process.env": JSON.stringify(process.env), // share  environment variables with esbuild
      };
      return options;
    },
    mdxOptions(options) {
      // this is the recommended way to add custom remark/rehype plugins:
      // The syntax might look weird, but it protects you in case we add/remove
      // plugins in the future.
      options.remarkPlugins = [
        ...(options.remarkPlugins ?? []),
        remarkGfm,
        remarkMdxImages,
      ];
      options.rehypePlugins = [...(options.rehypePlugins ?? [])];

      return options;
    },
  });

  return { code, metadata: getMetadata(slug, frontmatter) };
}

// at runtime, construct the mdx component only when the code prop changes
export default function SpecificNote({ code, frontmatter }) {
  const { title, lastmod } = frontmatter;
  const Component = useMemo(() => getMDXComponent(code), [code]);
  return (
    <>
      <MdxLayout lastmod={formatDate(lastmod)}>
        <h1>{title}</h1>
        <Component />
      </MdxLayout>
    </>
  );
}
```

## Conclusion

Learning about MDX and specifically how to process it on-demand forced me to dive in deep on how `@mdx-js/mdx`, `next`, and bundlers like `esbuild` and `webpack` work under the hood.

I learned about abstract syntax trees (great [AST visualizer](https://astexplorer.net/)), how to build a [simplified bundler](https://github.com/ronami/minipack), and came across new, to me, JavaScript [syntax](<(https://github.com/mdx-js/mdx/blob/main/packages/mdx/lib/run.js#L15)>) - `new AsyncFunction(String(file))(options)` - and JavaScript packages ([`string_decoder`](https://github.com/kentcdodds/mdx-bundler/blob/main/src/index.js#L3), [`Reflect`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Reflect/construct)) that I had to [parse](https://nodejs.org/dist/v8.9.4/docs/api/string_decoder.html) or [ask the community](https://github.com/mdx-js/mdx/discussions/2202).

Overall, it was a fun way to spend a week of my Christmas break.

{/* <TableOfContents toc={props.toc} /> */}


Owning my Content


4 common parts of modern-day websites.

## Transpiling/Compiling

- Modern websites use JavaScript.

- Browsers run the javascript using “just-in-time” compilers. These compilers turn developer-friendly JS code into computer-friendly machine code.

- However, JavaScript had evolved over the years, and not every browser is up-to-date with the latest syntax changes, especially when JavaScript added a bunch of features in 2015 and [transitioned](https://hacks.mozilla.org/2015/08/es6-in-depth-modules/) from version ES5 to ES6/ES2016.

- **Transpilers/Compilers**, like [Babel](https://babeljs.io/) or the fast Rust-written [SWC](https://swc.rs/), [transpile](https://en.wikipedia.org/wiki/ECMAScript#Transpiling) modern JS code into older JS code.

- **NextJS** uses the SWC compiler. They compile the code when you test it and build your project for production.

## Minifying

- “Minification is the process of removing unnecessary code formatting and comments without changing the code’s functionality. " ([source](https://nextjs.org/learn/foundations/how-nextjs-works/minifying): NextJS)

- NextJS uses the SWC minifier.

## Bundling

- Modern web apps break their code across multiple files and import/export things.

- Bundling is grouping these separate chunks of code into one file.

- There are a bunch of options: [esbuild](https://esbuild.github.io/), [rollup](https://rollupjs.org/guide/en/), [webpack](https://webpack.js.org/), SWC ([swcpack](https://swc.rs/docs/usage/bundling)), and [turbopack](https://turbo.build/pack).

- As of NextJS version 13, they’re using Rust-written turbopack. Prior they used webpack.

## Code Splitting

- Developers split webpage code across different files. Different files import similar and different things from one another.

- Bundlers bundles code for files that have similar imports, reducing the total number of bundles and redundant code that users would have to load.

## Compiling

- Compiling is taking developer-friendly code in one language (Javascript) and outputting it in a computer-friendly format, one that browsers understand.

- NextJS does compiling while you develop and test your code and when you build it for production. ([source](https://nextjs.org/learn/foundations/how-nextjs-works/compiling))

- NextJS has its compiler (SWC) written in Rust

## Rendering

- Rendering is showing the user the UI, the HTML styled w/CSS.

- Dynamic websites use JavaScript to make the UI interactive and render different things.

- The UI can be _pre-rendered_ or _client-side_ rendered.

- Pre-rendering can be done by a call to a server that returns the HTML, or by using **getStaticProps** in NextJS which gets all the things the webpage needs at **build time** so that NextJS can include the UI in the bundles.

- Client-side rendering is rendering on the fly, meaning users have to wait for the javascript to run and fill the webpage with HTML.


How modern websites work


Prettier is an _opinionated_ code formatter so it comes with its own belief on how code should be formatted and doesn't overwhelm you with ways to change it.

Prettier vs ESLint?

- Prettier formats your code (makes it look pretty).

- ESLint lints your code (finds errors such as broken HTML links).

## Install ([source](https://prettier.io/docs/en/install.html))

You can install and configure Prettier as an extension in your code editor, like the VS Code [prettier extension](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode).

OR you can install it as an npm package in your project which overrides your extension's setup and ensure your entire team formats code the same.

```bash

yarn add --dev --exact prettier

```

Prettier recommends using `--exact` so that your package version doesn't change as they release new updates.

## Configuring/customizing

Prettier comes with [good principles](https://prettier.io/docs/en/rationale.html) on how code should be formatted, but you may want to customize it a bit inside [`.prettierrc.json`](https://prettier.io/docs/en/configuration.html).

(_btw, 'rc' stands for "run commands" and the rc suffix is commonly used for any file that contains startup information for a program._)

Here's a bunch of prettier [defaults](https://prettier.io/docs/en/options.html):

```json
{
  "printWidth": 80, // 80 characters per line before they're wrapped

  "tabWidth": 2, // 2 spaces / indentation level

  "useTabs": false, // indents lines with spaces, not tabs

  "semi": true, // puts semicolons at the end of lines

  "singleQuote": false, // uses double quotes, not single quotes

  "trailingComma": "es5", // allows trailing commas in objects if valid by es5 standard

  "bracketSpacing": true, // turns {foo:100} into this { foo:100 }

  "bracketSameLine": false, // put the closing '>' for multiline HTML/React elements on a new line.

  "arrowParens": "always", // turns x => x into (x) => x

  "parser": "", // none, prettier automatically infers the language via it's file extension

  "proseWrap": "preserve", // won't wrap lines in markdown files

  "endOfLine": "lf", // prettier uses \n, not Window's \r\n for line endings

  "embeddedLanguageFormatting": "auto", // prettier tries to format code in markdown files

  "singleAttributePerLine": false // prettier won't change <p prop="" class=""> into a multiline HTML element
}
```

## Ignoring specific code

### Ignore with a [`.prettierignore` file](https://prettier.io/docs/en/ignore.html)

Create an ignore file (tells prettier which files/folders not to format) `.prettierignore`

```ignore

# Prettier ignores these by default

**/.git

**/.svn

**/.hg

**/node_modules

**/.git

**/.svn

**/.hg

```

### Ignoring within a file

Or ignore specific parts of files with comments

```javascript

// prettier-ignore

const  uglyMatrix = [[1,2,3],[4,5,6],[7,8,9],...,[1000,1001,1002]]



// prettier-ignore-start

const  uglyQuotes = 'word';

// prettier-ignore-end

```

## Using Prettier

### Using from the [CLI](https://prettier.io/docs/en/cli.html)

Check formatting on files:

```bash

npx prettier . --check

```

Actually format your files:

```bash

npx prettier . --write

```

(_Btw, `npx` lets you use locally installed packages in your CLI. If you don't have Prettier installed then it will temporally install it, but it's bad to use Prettier this way because then your code would be formatted differently for every new Prettier release._)

### Using from a script in `package.json`

You can create a script in `package.json`

```json
{
  "scripts": {
    "prettier-check": "prettier --check .",

    "prettier-write": "prettier --write ."
  }
}
```

### Using from the [API](https://prettier.io/docs/en/api.html) during runtime

You can format files/strings that contain code on the fly while your code is running. This is useful for things like generating sitemaps programmatically where `sitemap.xml` files must be formatted correctly.

Here's how to format strings of code: `prettier.format(source, options)`

It's important to set options with the correct [parser](https://prettier.io/docs/en/options.html#parser) specific to the language that the code/source is in, and any custom style options you want which will override the prettier defaults.

```javascript
const prettier = require("prettier");

prettier.format("foo ( );", { semi: false, parser: "babel" }); // -> "foo()"
```

If you want to format a file (not string), it's easier because Prettier can figure out the correct parser to use from the file extension.

```javascript
const prettier = require("prettier");

const text = fs.readFileSync(filePath, "utf8");

prettier.resolveConfig(filePath).then((options) => {
  const formatted = prettier.format(text, options);
});
```

## Integrating with ESLint

ESLint can format code too. This may conflict with how Prettier formats your code. The best advice is to extend your config file with the [`eslint-config-prettier`](https://github.com/prettier/eslint-config-prettier) package and follow the instructions on how to set it up:

```json
{
  "extends": [
    "eslint:recommended",

    "someOtherPackages",

    "prettier" // it's important to add this last
  ]
}
```

Then make sure to **run prettier before ESLint**.

## My personal setup

I like Prettier just how it ships so my setup isn't much.

1. Install [prettier](https://github.com/prettier/prettier):

```bash

yarn add --dev --exact prettier

```

3. Install [eslint-plugin-prettier](https://github.com/prettier/eslint-plugin-prettier#recommended-configuration) and integrate with ESLint

```bash

yarn add --dev eslint-plugin-prettier

```

```json
// .eslintrc.json

{
  "extends": ["other packages", "plugin:prettier/recommended"]
}
```

4. I create scripts to run them inside my `package.json` file

```json
{
  "scripts": {
    "prettier-write": "prettier --write .",

    "prettier-check": "prettier --check .",

    "lint-check": "eslint .",

    "lint-fix": "eslint . --fix",

    "clean": "prettier --write . && eslint . --fix"
  }
}
```

5. I create config and ignore files that simply contain the defaults so that I can easily check what those look like and modify them later if necessary.


My notes on prettier and a quick reference sheet.

Prettier Reference


ESLint is a JavaScript package that you can add to your project. It lets you set up rules to self-enforce high-quality and consistent code.

_It won't do anything unless you extend someone else's config file or explicitly turn on rules in your config file._

## Quick Start and basic knowledge needed

Install with ESLint's setup help

```javascript
npm init @eslint/config
```

OR clean install

```javascript
yarn add --dev eslint
```

Run it with the ESLint CLI:
`npx eslint [options] file.js [file.js] [dir]` OR create a script in your `package.json` file ([CLI source](https://eslint.org/docs/latest/user-guide/command-line-interface)).

### Configuring ESLint ([source](https://eslint.org/docs/latest/user-guide/configuring/))

ESLint's config file is `.eslintrc.js`. Assume it's blank:

```javascript
module.exports = {};
```

#### Rules ([source](https://eslint.org/docs/latest/user-guide/configuring/rules))

ESLint comes with a bunch of [rules](https://eslint.org/docs/latest/rules/) that you can manually turn on.

```javascript
module.exports = {
  rules: {
    eqeqeq: "error",
    quotes: ["warn", "double"],
    semi: "2",
  },
};
```

`"eqeqeq"` is the rule name, found on ESLint's docs or from a plugin's ruleset. After that is either the rule severity (`"error"`) or a list that has the severity followed by options you can set formatted as string/object/... it depends on the specific rule (`["warn", "double"]`). You can declare the severity with words or numbers: ` "0" = "off", "1" = "warn", "2" = "error"`.

You can also use specific rules in certain files with comments

```javascript
/* eslint eqeqeq: "off", curly: "error" */
```

Again, you see the format - _rule name_ : _rule value_

Or you can enable/disable rules in certain files

```javascript
/* eslint-disable no-console -- For entire file*/
console.log("won't show");

alert("foo"); // eslint-disable-line no-alert -- Specific rule, this line only

// eslint-enable-next-line -- All rules, next line only
console.log("hey");

/* eslint-disable-next-line no-alert */
alert("foo");
```

#### Plugins ([source](https://eslint.org/docs/latest/user-guide/configuring/plugins))

ESLint lets you define your own rules, or import someone else's rules that they created and published as an NPM package, eg. [`eslint-plugin-react`](https://github.com/jsx-eslint/eslint-plugin-react) which has linting rules for React and JSX. Or, [`eslint-plugin-react-hooks`](https://www.npmjs.com/package/eslint-plugin-react-hooks) which has linting rules for React Hooks.

```javascript
yarn add --dev eslint-plugin-react
```

```javascript
module.exports = {
	"plugins" ["react"],
	"rules" {
		"react/jsx-uses-vars": "error",
	}
}
```

Note, plugins require you to add the rules manually, if you don't want to do this then you should extend their config file.

#### Sharable Config Files ([source](https://eslint.org/docs/latest/user-guide/configuring/configuration-files#extending-configuration-files))

Manually adding rules and importing plugins gets tedious. Skip the work by extending someone else's config file which already includes plugins and rules declared. Eg. [`eslint: recommended`](https://eslint.org/docs/latest/user-guide/configuring/configuration-files#using-eslintrecommended) which comes with the `eslint` package or ones you've installed like [`eslint-config-airbnb`](https://github.com/airbnb/javascript/tree/master/packages/eslint-config-airbnb). Extending a config file also imports their plugins. Below, we extend the Airbnb config file which includes the react and react-hook plugins so we don't need to manually install or declare them anymore, aka we can ~~`"plugins": ["react"]`~~!

```javascript
module.exports = {
	"extends": [
		"eslint:recommended",
		"airbnb",
		"airbnb/hooks"
	],
	(...)
}
```

Instead of downloading a separate config file, `eslint-config-{pkgName}`, some plugins also come with a config file that you can use. Eg. [`eslint-plugin-react`](https://github.com/jsx-eslint/eslint-plugin-react). Using it is as simple as:

```javascript
module.exports = {
  extends: ["plugin:react/recommended"],
};
```

The `plugin:` syntax tells ESLint to look for the shared config file in the `eslint-plugin-react` package that we've installed.

## Advanced

These are sometimes not needed if you're extending from a config like [eslint-config-next](https://github.com/vercel/next.js/tree/canary/packages/eslint-config-next). Still helpful knowledge to have.

### Environments ([source](https://eslint.org/docs/latest/user-guide/configuring/language-options#specifying-environments))

Tell ESLint what environment your code runs in so that it knows the predefined global variables (e.g. the `"browser"` environment comes with vars like `window`).

```javascript
module.exports = {
    "env": {
        "browser": true,
        "node": true,
        "es6": true
    }
    (...)
}
```

### Parser Options

Tell ESLint what syntax you're using. ES3-ES5 is ESLint's default but ES6/ES2015 and beyond are modern-day JavaScript. Specifying `module` is important if you use `import`/`export` and `jsx` is important if you're using React.

```javascript
module.exports = {
    "parserOptions": {
        "ecmaVersion": "latest",
        "sourceType": "module",
        "ecmaFeatures": {
            "jsx": true
        }
    }
    (...)
}
```

### Custom rules

It gets fairly involved, here's [a good article](https://stevenpetryk.com/blog/custom-eslint-rules/) on it.

## My setup for a NextJS Website

### Config files I extend:

- [eslint:recommended](https://eslint.org/docs/latest/rules/)
- [eslint-config-next](https://github.com/vercel/next.js/tree/canary/packages/eslint-config-next) which includes in it:
  - Extends [`plugin:react/recommended`](https://github.com/jsx-eslint/eslint-plugin-react)
  - Extends [`plugin:react-hooks/recommended`](https://github.com/facebook/react/tree/main/packages/eslint-plugin-react-hooks)
  - Extends [`plugin:@next/next/recommended`](https://github.com/vercel/next.js/tree/canary/packages/eslint-plugin-next)
- [eslint-config-prettier](https://github.com/prettier/eslint-config-prettier) OR [eslint-plugin-prettier](https://github.com/prettier/eslint-plugin-prettier#recommended-configuration)
  - Add this last so that prettier can override other configs.
  - Run the prettier script (`prettier --write`) _before_ the ESLint script (`eslint --fix`) or else you'll get issues when the prettier format conflicts with your ESLint rules.
    - A package called [prettier-eslint](https://github.com/prettier/prettier-eslint) will do this for you (running Prettier then ESLint) but it's easy enough to do it yourself.

#### .eslintrc.json

```json
// .eslintrc.json
{
  "extends": [
    "eslint:recommended",
    "next",
    "next/core-web-vitals"
    "plugin:prettier/recommended",
  ],
  "rules": {
    "prettier/prettier": "error",
    "react/react-in-jsx-scope": "off",
    "react/prop-types": "off",
    "react/no-unescaped-entities": "off",
    "no-var": "error", // use let or const instead
    "require-await": "warn", // Disallow async functions which have no await expression
    "semi-style": ["warn", "last"], // semicolons must be at the end
    "semi": ["warn", "always"], // must use semi colons
    "quotes": [
      "warn",
      "double",
      { "allowTemplateLiterals": true, "avoidEscape": true }
    ], // must use double quotes
    "no-unused-vars": "warn" // don't have unused variables
  }
}
```

#### package.json

```json
//package.json
{
  "scripts": {
    "lint": "eslint .",
    "lint-fix": "eslint . --fix",
    "prettier-check": "prettier --check .",
    "prettier-write": "prettier --write .",
    "clean": "prettier --write . && eslint . --fix"
  },
  "devDependencies": {
    "eslint": "8.28.0",
    "eslint-config-next": "13.0.4",
    "eslint-config-prettier": "^8.5.0",
    "eslint-plugin-prettier": "^4.2.1",
    "prettier": "2.8.1"
  }
}
```


My notes on ESLint and a quick reference sheet.

ESLint Reference


Source: [NextJS tutorial on SEO](https://nextjs.org/learn/seo/introduction-to-seo/importance-of-seo)

(I'm focusing on the technical stuff. See this in action in my post, on [how I built my website](/notes/website-setup-2023))

## **SEO Pillars**

- **Technical** -- What the developer does
- **Creation** -- The content on the website
- **Popularity** -- Backlinks to your site from reputable sites

## **First off, fundamental knowledge**

As search engine web crawlers and users visit your web pages, they may encounter one of these HTTP status code messages. Some can affect your SEO!

import {
  Table,
  statusCodesTableRows,
  explicitTableRows,
  implicitTableRows,
} from "./tables";

<Table contents={statusCodesTableRows} />

## Explicit things you can do

There's a collection of obvious things developers do to improve SEO. Here they are.

<Table contents={explicitTableRows} />

## Implicit things you can do

These are strategies not so obvious that developers can do to improve SEO.

<Table contents={implicitTableRows} />

## Measuring Performance

[Web Vitals](https://web.dev/vitals/) are Google's terms referencing how they measure website performance.

The <ins>Core web vitals</ins> is a subset of web vitals that have 3 metrics that measure _loading_, _interactivity_, and _visual stability_ performance:

- LCP (largest contentful pain)
  - Time it takes for the largest element to load on the webpage
- FID (first input delay)
  - Time it takes for user's input to do something on the webpage
- CLS (cumulative layout shift)
  - Elements shifting after already being rendered

Google uses a website's core web vital scores to help _rank_ them.

- Scores are: **Good**, **Needs improvement**, **Poor**.

Tips to improve Web Vital Scores

- Optimizes images and fonts.
  - NextJS Image `<Image src={...} />`
  - Helps LCP and CLS score
- Use dynamic imports when possible `await import(...)`
  - Gets rid of warning to remove unused Javascript.
  - Helps FID score
- Defer loading third-party scripts until after the webpage content loads. `import Script from "next/script"`



Since my fall semester at UC Berkeley just ended, I thought what better time than now to redo my personal website. The last iteration of it was done nearly four years ago (also the last time I updated it 😬)! Going forwards, I want my website to be easier and more inviting to add to. This is what my website looked like in 2019 (source - [Wayback Machine](https://web.archive.org/web/20210909202641/colinsteidtmann.com/)):
![screenshot of my website from 2019](./2019-website.png)

So, how should I rebuild it? I could use pay for a site from Squarespace, Wix, or Wordpress, but then I'd be limited in ownership and creative freedom, and I wouldbe paying 10x what I pay now ($12/year). A week ago I stumbled across [MDX](https://mdxjs.com/), aka [Markdown](https://www.markdownguide.org/) with the ability to interweave JavaScript
in it. Markdown is a very friendly way to write and much more inviting to me, eg. I can write headers with `# Header` or italics with `*word*` instead of `<h1>Header</h1>` and `<em>word</em>`.Now, I'll write the steps I followed to set-up this site,
it uses NextJS, TailwindCSS, and MDX. Here's the high-level overview:

1. Install [NextJS](https://nextjs.org/)
2. Install [TailwindCSS](https://tailwindcss.com/)
3. Install [mdx-bundler](https://github.com/kentcdodds/mdx-bundler)
4. Do a bit of programming to set things up and look at other [projects](https://github.com/timlrx/tailwind-nextjs-starter-blog) for inspiration.

Now let's hop into some actual code.

## Installing things

Install NextJS. NextJs is a _framework_ for ReactJS, and ReactJS is a _library_ for JavaScript. React has a library of `functions()` to manipulate HTML and make pages more interactive and fun to use. NextJS then frameworks those functions in a way that helps developers create faster and better programmed websites.

```
yarn create next-app website-name
```

Next install TailwindCSS. TailwindCSS is a CSS _framework_ helps you apply a consistent style to your website but one that you'll still have full control of. Run the command below and then follow their specific [NextJS steps](https://tailwindcss.com/docs/guides/nextjs). (I included `@tailwindcss/typography` b/c we need that to style the markdown and `@tailwindcss/forms` b/c it might come in handy 🤷)

```
yarn add --dev tailwindcss postcss autoprefixer @tailwindcss/typography @tailwindcss/forms
```

The website part is done, let's now integrate MDX. We'll use mdx-bundler to _compile_ and _bundle_ the markdown/mdx files. Compiling will turn the markdown files into something parsable by browsers (HTML). Bundling will bundle together the JavaScript functions that MDX files can import from libraries or interweave in the content. Install mdx-bundler and it's dependencies (I included `gray-matter` b/c it will help in a following step):

```
yarn add mdx-bundler esbuild gray-matter
```

## Do some programming

Your project tree should look like this:

```
|--your-website-name
    |-pages
        |(...)
        |-index.js
    |-public
    |-styles
    |(...)
    |-tailwind.config.js
```

Now let's create a `content` directory to hold the MDX files and a `lib` and `lib/utils` directory to write the mdx helper code.
Inside `lib/utils` create `files.js` which will export a function that let's us iterate over all the mdx files in `content`:

```
// lib/utils/files.js
import fs from 'fs';
import path from 'path';

const pipe =
    (...fns) =>
        (x) =>
            fns.reduce((v, f) => f(v), x);

const flattenArray = (input) =>
    input.reduce((acc, item) => [...acc, ...(Array.isArray(item) ? item : [item])], []);

const map = (fn) => (input) => input.map(fn);

const walkDir = (fullPath) => {
    return fs.statSync(fullPath).isFile() ? fullPath : getAllFilesRecursively(fullPath);
};

const pathJoinPrefix = (prefix) => (extraPath) => path.join(prefix, extraPath);

const getAllFilesRecursively = (folder) =>
    pipe(fs.readdirSync, map(pipe(pathJoinPrefix(folder), walkDir)), flattenArray)(folder);

export default getAllFilesRecursively;
```

Next, inside `lib`, create `mdx.js` which will contain functions that use `mdx-bundler` to compile and bundle the the mdx files and returns HTML+_[frontmatter](https://jekyllrb.com/docs/front-matter/)_ (metadata we can include in the mdx files, e.g. date it was written):

```
// lib/mdx.js
import { bundleMDX } from "mdx-bundler";
import matter from "gray-matter";
import fs from "fs";
import path from "path";
import getAllFilesRecursively from "./utils/files";
// Remark packages
// Rehype packages

const root = process.cwd();
const contentDir = "content";
const componentsDir = "components";

// type is a subfolder in content, like 'blogs'
export function getFiles(type) {
    const prefixPaths = path.join(root, contentDir, type);
    const files = getAllFilesRecursively(prefixPaths);
    // Only want to return blog/path and ignore root, replace is needed to work on Windows
    return files.map((file) => file.slice(prefixPaths.length + 1).replace(/\\/g, '/'));
}

export function formatSlug(slug) {
    return slug.replace(/\.(mdx|md)/, '');
}

// type is a subfolder in contentDir, like 'blogs'
export async function getFileBySlug(type, slug) {
    const mdxPath = path.join(root, contentDir, type, `${slug}.mdx`);
    const mdPath = path.join(root, contentDir, type, `${slug}.md`);
    const source = fs.existsSync(mdxPath)
        ? fs.readFileSync(mdxPath, 'utf8')
        : fs.readFileSync(mdPath, 'utf8');

    // https://github.com/kentcdodds/mdx-bundler#nextjs-esbuild-enoent
    if (process.platform === 'win32') {
        process.env.ESBUILD_BINARY_PATH = path.join(root, 'node_modules', 'esbuild', 'esbuild.exe');
    } else {
        process.env.ESBUILD_BINARY_PATH = path.join(root, 'node_modules', 'esbuild', 'bin', 'esbuild');
    }

    const { code, frontmatter } = await bundleMDX({
        source: source,
        cwd: path.join(root, componentsDir),
        esbuildOptions: (options) => {
            options.loader = {
                ...options.loader,
                '.js': 'jsx',
            };
            return options;
        },
    });

    return {
        code,
        frontMatter: {
            slug: slug || null,
            fileName: fs.existsSync(mdxPath) ? `${slug}.mdx` : `${slug}.md`,
            ...frontmatter,
            date: frontmatter.date ? new Date(frontmatter.date).toISOString() : null,
        }
    };
}

// type is a subfolder in content, like 'blogs'
export async function getAllFilesFrontMatter(type) {
    const prefixPaths = path.join(root, contentDir, type);

    const files = getAllFilesRecursively(prefixPaths);

    const allFrontMatter = [];

    files.forEach((file) => {
        // Replace is needed to work on Windows
        const fileName = file.slice(prefixPaths.length + 1).replace(/\\/g, '/');
        // Remove Unexpected File
        if (path.extname(fileName) !== '.md' && path.extname(fileName) !== '.mdx') {
            return;
        }
        const source = fs.readFileSync(file, 'utf8');
        const { data: frontmatter } = matter(source);
        if (frontmatter.draft !== true) {
            allFrontMatter.push({
                ...frontmatter,
                slug: formatSlug(fileName),
                date: frontmatter.date ? new Date(frontmatter.date).toISOString() : null,
            });
        }
    });

    return allFrontMatter.sort((a, b) => dateSortDesc(a.date, b.date));
}
```

Now that the messy code is out of the way, let's move on to the fun stuff. Simply create some mdx file inside `content`, e.g. `hello.mdx`:

```
// content/hello.mdx

# Hello *world*
```

Then inside `pages/index.js` let's display the mdx content on the homepage (note the `prose` class name comes from `@tailwindcss/typography` and it helps us style the markdown content).

```
// pages/index.js
import { useMemo } from "react";
import { getMDXComponent } from "mdx-bundler/client";
import { getFileBySlug } from "../lib/mdx";

export async function getStaticProps() {
    const { code, frontMatter } = await getFileBySlug('', "hello");
    return { props: { code, frontMatter } };
}

export default function Home({ code, frontMatter }) {
    const Content = useMemo(() => getMDXComponent(code), [code]);
    return (
        <>
            <div className="prose mx-auto mt-8">
                <Content />
            </div>
        </>
    );
}
```

If everything works correctly, the homepage should show a white background with something like

# Hello _world_

written on it.


Setting up my website for 2023


Hi, I'm Colin.

I'm in my third year at UC Berkeley, majoring in Electrical Engineering and Computer Science, aka [EECS](https://eecs.berkeley.edu/). I _really love learning_, so outside of the two physical activities that I do (road biking 🚲 and walking my dog 🐾), I'm probably trying to learn something new 🙂.


About Me


import AllPosts from "./AllPosts.js";

A 39-day solo bike ride across America 🚲. YOLO

<AllPosts />


Colin Steidtmann

Notes