Improving the GitLab CI/CD log experience for jobs with multi-line commands has been a long-requested feature. With the latest release of GitLab and GitLab Runner, it's now easier to work with the log section for jobs with multi-line commands. In this post, we will describe the experience with the new feature, show you how to enable the new log output in your pipelines, and discuss key points regarding CI/CD script execution and log output in various shells, such as Bash and Powershell.
Overview of multi-line commands
First, it’s helpful to describe what we mean by a CI job with multi-line commands. In GitLab CI the script keyword is used to specify commands to execute for a CI job. In the example below, the build-job has a single command, a basic echo statement, to execute in the script block.
## A pipeline with a single line command in the script block for the build-job
build-job:
stage: build
script:
- echo "this is the script to run for the build job"
If you were to run this pipeline, then the log output in the UI would display as follows: Line 17 - GitLab CI automatically generates a log entry for the command that you specify in the script block. Line 18 - This is the output of the command that was executed.
Now as you can imagine, the script that you define in the script block will likely be more complex than the example provided and could very well span multiple lines in the CI/CD pipeline file.
## A pipeline with a multi-line command in the script block for the build-job
build-job:
stage: build
script:
- |
echo "this is a multi-line command" # a simple echo statement
ls
If you were to run this pipeline, then the log output in the UI would display as follows:
Line 17 - As in the previous example, GitLab CI automatically generates a log entry for the command that you specify in the script block. You will notice that line 17 only includes the first command in the script block. This makes it more difficult to debug an issue with script execution as you will need to refer back to the source pipeline file to see exactly what script was executed.
So what’s new?
Starting in GitLab 16.7 and GitLab Runner 16.7, you can now enable a feature flag titled FF_SCRIPT_SECTIONS, which will add a collapsible output section to the CI job log for multi-line command script blocks. This feature flag changes the log output for CI jobs that execute within the Bash shell.
Line 17: Unlike the previous examples, the first thing you will notice in the screenshot above is that by default the log entry for the multi-line command is collapsed by default.
Single-line commands do not display in a collapsible element.
For multi-line scripts the multi-line command is now a collapsible element, so now, when you uncollapse the log entry for line 17, then the log will display all of the commands that were executed in the script block.
There is also the custom collapsible section feature, which in combination with this new multi-command output capability does provide you additional flexibility for displaying log output in the UI. Here is how you can use the two features to change the log output.
## A pipeline with a multi-line command in the script block for the build-job
variables:
FF_PRINT_POD_EVENTS: "true"
FF_USE_POWERSHELL_PATH_RESOLVER: "true"
FF_SCRIPT_SECTIONS: "true"
collapsible_job_multiple:
stage: build
script:
- |
echo "{
'test': 'data',
'test2': 'data2',
}"
- |
echo "{
'test': 'data',
'test2': 'data2',
}"
- echo -e "\033[0Ksection_start:`date +%s`:my_first_section\r\033[0KHeader of the 1st collapsible section"
- echo 'this line should be hidden when collapsed'
- |
echo "{
'test': 'data',
'test2': 'data2',
}"
- echo -e "\033[0Ksection_start:`date +%s`:second_section\r\033[0KHeader of the 2nd collapsible section"
- echo 'this line should be hidden when collapsed'
- echo -e "\033[0Ksection_end:`date +%s`:second_section\r\033[0K"
- echo -e "\033[0Ksection_end:`date +%s`:my_first_section\r\033[0K"
If you were to run this pipeline with the FF_SCRIPT_SECTIONS feature flag set to false, then the log output would be as depicted in the following screenshot.
But, if you were to run this pipeline with the FF_SCRIPT_SECTIONS feature flag set to true, then the log output would be as depicted in the following screenshot.
What about other shells?
As of the 16.7 release, the collapsible output section in the CI job log for multi-line command script blocks is only visible for CI/CD jobs that are executed with the Bash shell. CI/CD jobs executed with Powershell is not currently supported. We plan to add this capability in a future release.
What are our future plans?
A few features are still needed to improve the CI/CD job log output, and the timestamp for each log line is one of them. This addition will add missing features such as command/section duration.
To learn more about GitLab CI/CD features, refer to the official CI/CD documentation.
Disclaimer: This blog contains information related to upcoming products, features, and functionality. It is important to note that the information in this blog post is for informational purposes only. Please do not rely on this information for purchasing or planning purposes. As with all projects, the items mentioned in this blog and linked pages are subject to change or delay. The development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab.
