At Stripe, our product docs are designed to feel like an application rather than a traditional user manual. For example, we incorporate a user's own API test key into code samples, making it possible to copy and paste code that seamlessly works with the user's own account. We have client-side interactivity, like checklists and collapsible sections. We tailor the content to the individual user, conditionally displaying content based on their location or the Stripe features they use. These features result in a high-quality user experience that reduces friction and contributes to the success of developers.

For these capabilities to have the desired impact we have to make it easy for writers to use them in their content. Delivering a good user experience without compromising the authoring experience required us to develop an authoring format that enables writers to express interactivity and simple page logic without mixing code and content.

Over several years, we learned how to balance interactivity, customization, and authoring productivity while undertaking a major overhaul of our documentation platform.

Past is prologue

To understand how we got here it's important to understand where we started. The legacy documentation platform that we replaced was a monolithic Ruby application built with ERB templates and Sinatra routing. The content freely mixed HTML, Markdown, Ruby, and ERB helper functions.

Mixing code and content provided a natural way to programmatically tailor the docs to the developer, but it posed serious challenges to quality and maintainability when the body of content grew to hundreds of pages. Alongside the technical burden of maintaining code within the content, the behavior of the code can make the content itself harder to understand and manipulate safely, particularly when used by many different teams with different objectives, timetables, and areas of expertise. Content authoring effectively became software development, and with that became subject to the same technical complexity and overhead. 

We wanted to introduce new content formats with significantly more interactivity and more sophisticated frontend logic, but we knew that the limitations of a code-first approach would prevent us from using these features widely. For example, our integration builder format, which was originally created as a React application with content authored in JSON, became much easier for technical writers to reproduce and maintain when it was migrated to use Markdoc for the authoring experience.

Designing Markdoc

When we began building our current documentation platform, we wanted to simplify our authoring experience by adopting an intuitive format like Markdown. Although Markdown is significantly easier to read and reason about than ERB templates, its simplicity also imposes limitations that make it challenging to use for rich content like our product docs.

Markdown is a relatively flat format that isn't designed to express complex structure or hierarchy. It offers a small number of formatting features and provides limited control over presentation. It does not have exotic templating features like support for custom page logic, variables, conditionals, or content reuse. Markdown's enduring success and relative ubiquity are largely due to its intentionally narrow scope and the restraint exercised in its design. It is easy and enjoyable to use because it prioritizes readability and leans heavily on intuitive plain-text authoring conventions.

Our custom authoring format, called Markdoc, was designed to decouple code and content while enforcing proper discipline at the boundaries. Instead of allowing each page to be treated like an open-ended application, it imposes constraints on styling and programming, providing prescriptive rails for content extensibility. It extends Markdown with custom syntax that meets the needs of our documentation platform without sacrificing Markdown's simplicity, familiarity, or ease of use for writing prose. Following the ethos and design sensibility of Markdown, Markdoc keeps the overall surface area of new features small by adding a few highly-composable primitives that can be used together to express all the functionality we need.

Markdoc provides an extensible system for defining custom tags that can be used seamlessly in Markdown content. Using the custom tag syntax, we're able to support features like conditional content, content inclusion, and variable interpolation.

The features we decided to leave out of Markdoc in order to protect content maintainability are a critical aspect of its design. For example, when deciding what built-in flow control to include in Markdoc, we deliberately chose not to include looping. We wanted to discourage writers from performing procedural content generation from inside of a document, forcing them to move it outside of the system for better encapsulation. We also decided to leave out variable assignment in order to ensure that the content is fully stateless, thus eliminating an entire class of potential bugs.

I like Markdoc because it lets us still do anything we want with code in the docs without bogging down the content authoring experience. If we need some new component, designers and engineers can whip that up. So as a writer, I can work in the docs content and stay focused.

React integration

Markdoc has a modular rendering system that supports multiple output formats. Using Markdoc’s React renderer, a Markdoc document can be rendered to a React virtual DOM. Custom Markdoc tags can be configured to output React components, passing through tag attributes as React props. Markdoc also supports assigning custom React components to standard Markdown document nodes such as headings and paragraphs.

Defining custom Markdoc tags that output React components makes it possible to include interactive features, like tab switchers and collapsible content sections, inside of documents. Using custom tags to express these features helps create a writer-friendly interface for the functionality.

The React ecosystem also has a wealth of useful and interesting libraries that we can incorporate into our documentation to enrich presentation. For example, we're using the React Flow library to create interactive diagrams in our documentation. We defined a set of Markdoc tags for expressing the contents of a diagram, making it easy for writers to build beautiful and consistent visual representations of APIs and technical concepts from a set of composable elements.

Unlike static images, the diagrams that are built with React Flow can easily incorporate interactivity and clickable links. They are also easier to localize and can be restyled universally.

Moving our documentation frontend to React was an important goal of our platform overhaul. Stripe already used React across many parts of the user experience, including our API reference docs and user dashboard. Enabling integration and cross-pollination between those surfaces and the product docs opens up a lot of exciting opportunities for future innovation, like showing API reference overlays when the reader hovers their cursor over a function or parameter in a code example. Sharing a common set of components from Stripe's internal design pattern library helps improve cohesion.

React also offers some compelling technical advantages. The implementation of interactive frontend components in our legacy stack was split between markup ERB templates and logic written in JavaScript which made it difficult to properly encapsulate, extend, and reuse functionality—a set of problems that modern component-based frontend frameworks address in a more satisfying way.

Markdoc comes with two distinct React renderers: a renderer that dynamically builds the React virtual DOM tree on the client side, and a static renderer that transpiles the document to JavaScript code. We use the dynamic renderer in our documentation platform at Stripe, but the static renderer is useful in cases where you want to treat a piece of Markdoc content as though it is a React component or JavaScript module. For example, the static renderer makes it possible to implement a Markdoc loader for Webpack in only five lines of code.

Modular rendering

Alongside the React renderers, Markdoc also includes a string-based HTML renderer that can be used for conventional server-side rendering or integration with standards-based Web Components. Markdoc's modular rendering architecture makes it possible for third parties to build custom renderers for additional frameworks and systems.

Markdoc content is entirely agonistic with respect to the technology used to present the rendered document. Fully decoupling rendering from the document format gives us the flexibility to present the same content in radically different ways in the future—like incorporating it into a native mobile application or generating a print-ready output format such as a PDF. I even used Markdoc to make the slide deck for my presentation at the Write the Docs conference back in 2020. The Markdoc community has already started bringing support to other frameworks, including Vue and Svelte.

Ensuring that rendering implementation details don't bleed into the content also helps to improve the authoring experience, avoiding complexity and simplifying maintainability.

Documentation as data

Markdoc's fully declarative syntax parses to an Abstract Syntax Tree (AST), a data structure that represents the content of the document. We can take advantage of the AST to perform advanced static analysis and programmatically manipulate our content.

Markdoc lets us treat our documentation like data, writing simple scripts to programmatically inspect the content. If we want to perform tasks like identifying all of the fenced code blocks that contain a specific string or all of the places where we have a heading nested inside of a callout, we can do that robustly with the AST instead of relying on text scraping and regular expressions.

We are building automated refactoring tools that use the AST, making it possible to perform complex edits across our entire body of content with a higher degree of robustness than old-fashioned find-and-replace.

One of the most important ways that we use the AST today is for validation. For every Markdoc tag and document node type, there's a schema definition specifying the names and types of the attributes it accepts, what kind of document nodes can be nested inside of it as children, and other relevant metadata. The Markdoc validator uses this information to verify the correctness of a given Markdoc document.

Schemas can also include arbitrary logic that analyzes the document nodes and returns custom errors. We use this to support features like link validation, checking to make sure that every link between pages within our documentation points to a valid route. It can also be useful for enforcing certain style guidelines that relate to the document structure, like preventing authors from using the wrong heading levels in certain places.

We run the Markdoc validator in our continuous integration system to ensure correctness at build time, but we also have an internal Visual Studio Code extension that exposes validation errors in real time while the user is typing.

Markdoc makes it easy for me to build rich, interactive experiences around documentation, then surface that capability to other authors through a simple declarative interface.

Under the hood

Markdoc's parser is written in JavaScript and built on top of a popular open-source Markdown library called markdown-it. Markdoc is relatively lightweight—the markdown-it library is its only direct dependency. It is intended to run in Node.js and similar server-side JavaScript environments, but it can also be bundled for use in the browser.

Markdoc uses markdown-it as a tokenizer, building its AST from the array of tokens emitted by markdown-it. Parsing logic for Markdoc's custom tag syntax is generated from a peg.js grammar and integrates with markdown-it via a plugin.

Markdoc has its own dedicated rendering architecture rather than relying on markdown-it to generate its output. Developing an independent rendering system was necessary in order to handle Markdoc's custom tags and support multiple output formats.

Markdoc rendering is performed in several phases. First, the variable resolution step converts all of the variables in the document into their corresponding values. Next, the transformation step recursively walks through the document nodes in the AST and uses the node and tag schema definitions to generate a tree of renderable document nodes—a data structure that corresponds with the shape of the rendered output. Finally, the tree of renderable document nodes is passed into the desired renderer, which emits the actual rendered output.

Markdoc document's AST can be serialized to JSON and cached for later use, improving performance by obviating the need to parse the document every time it is rendered. Our product documentation platform at Stripe maintains an in-memory cache of the AST at runtime, but we are considering moving to an architecture where we serialize the AST at build time in order to eliminate runtime Markdoc parsing entirely.

When I first started using Markdoc at Stripe, I was delighted by how easy it was to structure docs exactly as I envisioned them. With other authoring tools, useful visual elements like collapsible sections, asides, tabs, tables, multiple-language code samples, and many more often required heavy customization or new development. With Markdoc, I have a full palette ready to use. After using Markdoc, it's hard to imagine going back to another authoring tool.

May the source be with you

Our team at Stripe spends a lot of time thinking about the authoring experience and how to get it right. In many ways, Markdoc is the embodiment of our obsession with building a better authoring experience. It's our way of bottling up everything we have learned about this topic and sharing it in a reproducible way.

After migrating all of our content to Markdoc and seeing the advantages fully realized in production, we set out to make Markdoc available under an MIT license so that others could benefit from our efforts.

We released Markdoc to the public in May, publishing a package on npm. We also published a draft specification that formally describes the Markdoc tag syntax, with the aim of making it easier for developers to incorporate support for Markdoc tags into other Markdoc parsing libraries.

Markdoc is hardly the final word on content authoring, but we hope that our contribution to the dialog will inspire others and help elevate discussion about the importance of the authoring experience in documentation.

Interested in using Markdoc at work? Let us know how we can help.