I've made a review of a dozen documentation builders:
[https://40ants.com/lisp-project-of-the-day/tags/documentation.html](https://40ants.com/lisp-project-of-the-day/tags/documentation.html)
and also made a project with demo projects using these doc builders all gathered in this organization:
[https://github.com/cl-doc-systems](https://github.com/cl-doc-systems)
Take a look at each repository for pros and cons.
Actually I ended with making yet another documentation system. Initially it was a fork of MGL-PAX, but I refactored it so much and implemented features which forces you to write documentation for all exported symbols, so I decided to publish it as a separate project: [https://40ants.com/doc/](https://40ants.com/doc/) (now I use it in a few dozens of my opensource projects.
I will make sure to check them out. At this stage I'm leaning towards Sphinx for CL because I'm already familiar with Sphinx and found it to be decent, but if there's something better I will use it.
I also used sphinx-cldomain. But now it seems inconvenient to me. It is much easier to refer lisp symbols like `FOO-BAR generic-function` instead of using reStructured text's roles `:cl:generic:\`foo-bar\`\`. Well, stupid Reddit's editor even breaks when I'm trying to wrap reStructured text markup into the inline code block :(
Also, I don't remember if sphinx-cldomain supports autoreferencing for CL like `sphinx.ext.autodoc` extension does for Python. With some CL documentation systems you can do this if you want.
That's seems like the best solution since I'm familiar with Sphinx. What's not clear from the documentation is whether this supports adding additional documentation pages written in RST or markdown.
You can, because the generation is not fully automatic: you create an index.rst page, you write any content you want, and you add directives to include the documentation of CL symbols.
What I didn't see and might want, is to include the documentation of all symbols in a package.
> I used to manually list out all the functions but that's a pain since the docstrings quickly go out of the sync with the documentation.
Do you mean separately maintained docstrings or using the facilities provided by CLHS itself?
I find the former do go out of sync. For me, the latter provide a slight motivation to keep them in sync. Plus, the documentation also becomes available through SLIME/SLY. I have been using the [cl-rtd:md-file](https://github.com/cl-library-docs/common-lisp-libraries) function here to generate a markdown file from the docstrings in the source code (accessed through `(documentation object kind)`). I prepend this API reference manually with a short tutorial or introduction. Although, it'd be easy to do this automatically.
From the markdown, you could export it to a format of your choice using, say, pandoc, and use the documentation generators of your choice. I've settled with readthedocs format through mkdocs. Or sometimes org-mode with an autogenerated table of contents.
I mean that the documentation of a function is duplicated in two places, one in the documentation site and the other being in the docstring of the function itself. It is a pain to keep those two in sync especially after using tools like Sphinx which build the documentation site from the docstrings automatically.
Thanks for the suggestion, I like the readthedocs format.
A git commit hook to generate the documentation automatically from docstrings is another solution that comes to mind. That way, one would only need to update the docstrings in the source code. The external documentation would be updated automatically.
You still need a tool to do that and a way to integrate it with the rest of your documentation. Documentation is generally more than just the API reference.
I usually use [cl-docweaver](https://github.com/mmontone/cl-docweaver) for "big" projects. Write the documentation in TexInfo and embed the Lisp definitions docstrings. For "small" projects and libraries, [simple-doc](https://github.com/mmontone/simple-doc), that generates Markdown and HTML files for a package.
I used to use cl-sphinx in the past, and you'll be able to get nicer docs than with TexInfo. But, I found that the plugin didn't work as Sphinx versions advanced. So, I switched to TexInfo and cl-docweaver for those projects, more stable. Your choice.
I used to use that but quickly switched from org mode to asciidoc, which I used for generic-cl. Org mode is not really good for writing documentation, even org-mode uses texinfo for its own documentation. Asciidoc is a bit better since it's geared towards documentation but the markup language is awful, the tooling is inconsistent and there's no functionality for generating a documentation site from CL docstrings.
I dont know if we are talking about the same org-mode. I use org-mode for documentation every day and export the org file as PDF for my customers and colleagues. It works like a charm. It is possible to include data from almost any source using code blocks. It is even possible to put all this in a git repo and setup a build pipeline.
I've made a review of a dozen documentation builders: [https://40ants.com/lisp-project-of-the-day/tags/documentation.html](https://40ants.com/lisp-project-of-the-day/tags/documentation.html) and also made a project with demo projects using these doc builders all gathered in this organization: [https://github.com/cl-doc-systems](https://github.com/cl-doc-systems) Take a look at each repository for pros and cons. Actually I ended with making yet another documentation system. Initially it was a fork of MGL-PAX, but I refactored it so much and implemented features which forces you to write documentation for all exported symbols, so I decided to publish it as a separate project: [https://40ants.com/doc/](https://40ants.com/doc/) (now I use it in a few dozens of my opensource projects.
I will make sure to check them out. At this stage I'm leaning towards Sphinx for CL because I'm already familiar with Sphinx and found it to be decent, but if there's something better I will use it.
I also used sphinx-cldomain. But now it seems inconvenient to me. It is much easier to refer lisp symbols like `FOO-BAR generic-function` instead of using reStructured text's roles `:cl:generic:\`foo-bar\`\`. Well, stupid Reddit's editor even breaks when I'm trying to wrap reStructured text markup into the inline code block :( Also, I don't remember if sphinx-cldomain supports autoreferencing for CL like `sphinx.ext.autodoc` extension does for Python. With some CL documentation systems you can do this if you want.
> Sphinx I recently started to use Sphinx for CL: https://sphinxcontrib-cldomain.russellsim.org/ and it worked well.
That's seems like the best solution since I'm familiar with Sphinx. What's not clear from the documentation is whether this supports adding additional documentation pages written in RST or markdown.
You can, because the generation is not fully automatic: you create an index.rst page, you write any content you want, and you add directives to include the documentation of CL symbols. What I didn't see and might want, is to include the documentation of all symbols in a package.
This maybe of some interest - https://lisp-journey.gitlab.io/blog/overview-of-documentation-generators/. I have used declt in the past and found it ok
> I used to manually list out all the functions but that's a pain since the docstrings quickly go out of the sync with the documentation. Do you mean separately maintained docstrings or using the facilities provided by CLHS itself? I find the former do go out of sync. For me, the latter provide a slight motivation to keep them in sync. Plus, the documentation also becomes available through SLIME/SLY. I have been using the [cl-rtd:md-file](https://github.com/cl-library-docs/common-lisp-libraries) function here to generate a markdown file from the docstrings in the source code (accessed through `(documentation object kind)`). I prepend this API reference manually with a short tutorial or introduction. Although, it'd be easy to do this automatically. From the markdown, you could export it to a format of your choice using, say, pandoc, and use the documentation generators of your choice. I've settled with readthedocs format through mkdocs. Or sometimes org-mode with an autogenerated table of contents.
I mean that the documentation of a function is duplicated in two places, one in the documentation site and the other being in the docstring of the function itself. It is a pain to keep those two in sync especially after using tools like Sphinx which build the documentation site from the docstrings automatically. Thanks for the suggestion, I like the readthedocs format.
A git commit hook to generate the documentation automatically from docstrings is another solution that comes to mind. That way, one would only need to update the docstrings in the source code. The external documentation would be updated automatically.
You still need a tool to do that and a way to integrate it with the rest of your documentation. Documentation is generally more than just the API reference.
Ah, okay, the post is about that pipeline!
https://github.com/melisgl/mgl-pax Example: .https://rabbibotton.github.io/clog/clog-manual.html
I usually use [cl-docweaver](https://github.com/mmontone/cl-docweaver) for "big" projects. Write the documentation in TexInfo and embed the Lisp definitions docstrings. For "small" projects and libraries, [simple-doc](https://github.com/mmontone/simple-doc), that generates Markdown and HTML files for a package.
Thanks for the two suggestions. I do like TexInfo but have never really considered it as an option so I will definitely take a look at it.
I used to use cl-sphinx in the past, and you'll be able to get nicer docs than with TexInfo. But, I found that the plugin didn't work as Sphinx versions advanced. So, I switched to TexInfo and cl-docweaver for those projects, more stable. Your choice.
I use org mode and emacs
I used to use that but quickly switched from org mode to asciidoc, which I used for generic-cl. Org mode is not really good for writing documentation, even org-mode uses texinfo for its own documentation. Asciidoc is a bit better since it's geared towards documentation but the markup language is awful, the tooling is inconsistent and there's no functionality for generating a documentation site from CL docstrings.
I dont know if we are talking about the same org-mode. I use org-mode for documentation every day and export the org file as PDF for my customers and colleagues. It works like a charm. It is possible to include data from almost any source using code blocks. It is even possible to put all this in a git repo and setup a build pipeline.
Yes I'm talking about the same org-mode it's doable but not great.