Building simple documentation projects with Foliant is usually straightforward. But Foliant is a powerful, customizable, and very flexible tool, capable of turning your most complex ideas into beautiful documents. If you understand exactly what you want to achieve, you can formalize it at the project config level, and Foliant will perform your task efficiently and precisely.
But sometimes it is difficult to configure all preprocessors and backends properly in one go. Some settings are pretty subtle and some preprocessors are quite complicated. The order of applying the preprocessors matters. Some preprocessors may work unexpectedly when paired with others. Fetching data from external sources may also become a bottleneck. The list goes on.
Fortunately, Foliant will not ask you to diagnose problems with the car engine without opening the hood. Foliant provides advanced diagnostic facilities such as:
- detailed event logging in the debug mode;
- the pre backend which does nothing, i.e. just returns the preprocessed Markdown;
- an option to keep the temporary working directory for further analysis.
Notes on Docker Use¶
In practice, Foliant is more commonly used with Docker.
Here's a tip for debugging Foliant projects with docker.
It’s useful to add one more service to your project's default
docker-compose.yml. We will call it
bash and it will run containers with an interactive shell:
version: '3' services: foliant: build: context: ./ dockerfile: ./Dockerfile volumes: - ./:/usr/src/app/ bash: build: context: ./ dockerfile: ./Dockerfile volumes: - ./:/usr/src/app/ entrypoint: /bin/bash
Now you can run a container based on the project’s image with an interactive shell. To open the shell for root, run:
$ docker-compose run --rm bash
To open shell for a user with the same user ID and group ID as your current user on the host machine:
$ docker-compose run --user="$(id -u):$(id -g)" --rm bash
All debugging approaches which we will discuss next are represented as native Foliant commands, but they are applicable for the Docker way too. Just start your commands with
docker-compose run --rm or
docker-compose run --user="$(id -u):$(id -g)" --rm to run them within Docker containers.
foliant make ... command runs Foliant in the regular logging mode. In this mode, Foliant and its extensions will only log events with levels of
warning. Note that some preprocessors may generate a lot of specific warnings which may or may not indicate that something went wrong. These messages are usually worth studying anyway though.
The new log file is created for each build, unless there were no errors and warnings. The logs are stored by default in the project root under the name
<unix timestamp of the build>.log, for example,
1628582527.log. With such a naming convention the log file for the latest build will always be last in alphabetical order. The location of the log files may be customized by the
--logs|-l command-line option.
Foliant provides the
-d command-line option which enables the debugging mode. In this mode, Foliant and its extensions log not just events with levels
warning, but also events with levels
info. The amount of information you will get from such events depends on the implementation of a particular extension. Complex preprocessors like Includes usually log their actions in great detail. The messages of the
info level are usually informative: they may mark the beginning or end of some preprocessor’s work, for example. The messages of the
debug level generally show the status of atomic operations, for example, reading data from a certain file. These messages often contain the values of the variables which are important in the current context: paths to files, external commands that are called, etc. But to make sense of these values prepare to get your hands dirty, or, in other words, read and understand the code of the corresponding extension.
Here's an example of a command that tells Foliant to build PDF with Pandoc in debug mode:
$ foliant make pdf --with pandoc --debug
Each log is a text file that contains a number of lines (records). Each record represents a single event and consists of 4 separate fields:
- date and time of the event registration;
- context (module name) in which the event was registered;
- event log level: one of
- message text that explains the essence of the event.
For example, the first record of a log usually looks like that:
2020-06-25 09:40:54,419 | flt | INFO | Build started.
flt in the second field means Foliant itself (Foliant Core).
The context is hierarchical. The following record represents an event that is registered in the Includes preprocessor which was implicitly called by the Flatten preprocessor, which was implicitly called during project build by Pandoc backend.
2020-06-25 09:40:54,678 | flt.pandoc.flatten.includes | DEBUG | Processing Markdown file: /usr/src/app/__folianttmp__/__all__.md
In the next example, Pandoc backend logs the external command that is called to build needed target:
2020-06-25 09:40:54,684 | flt.pandoc | DEBUG | PDF generation command: pandoc --template="/foliant_stuff/pandoc_templates/tex_templates/main.tex" --output "My_Awesome_Project-1.0-2020-06-25.pdf" --variable title="My Awesome Project" --variable version="1.0" --variable subtitle="Description Of My Awesome Project" --variable logo="/foliant_stuff/pandoc_templates/logos/logo.png" --variable year="2020" --variable title_page --variable toc --variable tof --pdf-engine=xelatex --listings -f markdown __folianttmp__/__all__.md
If you suspect that the command executes wrong, you can run it directly in an interactive shell and study the results.
Detailed logging in debug mode allows you to quickly localize problems zooming in from Foliant itself to a specific Foliant extension, a specific Markdown source file, or a specific line of code. This takes effort but with practice allows one to solve complex problems in minimal time.
Each Foliant backend takes preprocessed Markdown content and passes it to an external command (see Architecture And Basic Design Concepts). For debugging backends it's essential to see the content which the backend actually gets.
During the build source files of Foliant project are copied to a temporary working directory. By default, it is called
__folianttmp__/ and located in the “root” directory of the project. Source Markdown files of the project are kept unchanged during the build. Any transformations are applied only to the files located in the temporary working directory.
The pre backend¶
Foliant Core provides the built-in backend
pre which does nothing. More precisely, this backend makes the
pre target. The
pre target is obtained simply by copying the temporary working directory to a subdirectory inside the project root as the result of the build.
pre target is essentially the content that comes after all preprocessors are applied, but before any backend (other than
pre) is called.
Determining the cause of the problem
pre backend is convenient to determine the stage of a build which causes problems:
- configuration stage (reading the configuration file),
- preprocessing stage (transforming the Markdown content with extensions)
- or backend stage (producing the output format).
If you build a
pre target and the results seem fine, then there is a problem with your backend and you have to debug that. You may start with the keep-tmp which we will discuss next.
If the problem persists in the results, produced by
pre, then it's one of the preprocessors causing trouble or the configuration parser not working properly. In this case, it's a nice idea to stick with the
pre target during your experiments so you won't need to wait each time for the backend to complete producing the target while you are debugging the build.
To build a Foliant project to the
pre target, run the command:
$ foliant make pre
Keeping the project sources¶
In addition to the
pre backend, Foliant Core supports the
-k command-line option. By default, the temporary working directory (
__folianttmp__) is removed after the project build. But if the
-k option is specified, the temporary working directory will stay in the project root after build.
This directory will contain the files that are modified by all preprocessors and the chosen backend.
If you have determined that the backend causes issues, pre won't help you anymore. Run the build with
-k argument and study the working dir. If it seems fine, then the problem may be with the command that the backend runs to convert Markdown to a target format. Time to study logs!
The following command tells Foliant to build PDF with Pandoc, keeping the temporary working directory after build:
$ foliant make pdf --with pandoc --keep_tmp
Killing Two Birds With One Stone¶
Now you know what debugging facilities are provided by Foliant. But we strongly recommend you make it a rule to start debugging Foliant projects with one universal shell command:
$ foliant make pre --debug
This command tells Foliant to build the
pre target in the debug mode. And this is a very effective way to get closer to understanding what is wrong with your project.