---
title: "checktor in Continuous Integration"
vignette: >
  %\VignetteIndexEntry{checktor in Continuous Integration}
  %\VignetteEngine{quarto::html}
  %\VignetteEncoding{UTF-8}
knitr:
  opts_chunk:
    collapse: true
    comment: '#>'
---

```{r}
#| label: setup
#| include: false
library(checktor)
```

<style>
.ct-fig{display:block;margin:1.5rem auto;max-width:100%;height:auto}
.ct-dark{display:none}
@media (prefers-color-scheme:dark){.ct-light{display:none}.ct-dark{display:block}}
[data-bs-theme=light] .ct-light{display:block}
[data-bs-theme=light] .ct-dark{display:none}
[data-bs-theme=dark] .ct-light{display:none}
[data-bs-theme=dark] .ct-dark{display:block}
</style>

You already run `checktor` on your package. The trouble with running it by hand
is that you do it when you remember to, which is not the same as every time it
matters. A pipeline has no such lapses, running the check on every push, for
every contributor, whether or not anyone remembered to.

Moving the check into continuous integration (CI) turns a good habit into an
unconditional one, in about as many lines of YAML as it takes to describe.

## The one function CI needs

Everything in this article rests on a single function. `checkup()` runs the
full diagnosis and collapses it to one verdict: `TRUE` if the package is clean,
`FALSE` if a policy or robustness finding wants attention. Opinion findings are
still reported, but they do not fail it.

```{r}
#| label: scenario
# A throwaway package that deliberately uses T/F instead of TRUE/FALSE
bad_pkg <- example_diagnose_scenario("code_examples/tf_usage_bad.R",
                                     show_content = FALSE)

checkup(bad_pkg)
```

A healthy package returns `TRUE`. That is the entire contract, and it is all a
build needs to decide whether to go green or red.

## A GitHub Actions job

`checkup()` is designed to be the last word in a shell one-liner. Drop this file
in at `.github/workflows/checktor.yaml` and you have an extra-CRAN gate on every
push and pull request:

```yaml
name: checktor

on:
  push:
    branches: [main, master]
  pull_request:

jobs:
  checktor:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7

      - uses: r-lib/actions/setup-r@v2
        with:
          use-public-rspm: true

      - uses: r-lib/actions/setup-r-dependencies@v2
        with:
          extra-packages: any::checktor

      - name: Run extra-CRAN checks
        run: if (!checktor::checkup()) quit(status = 1)
        shell: Rscript {0}
```

The job checks out your package, installs `checktor`, and exits non-zero the
moment `checkup()` disagrees with you.

> `any::checktor` installs the release from CRAN. To run the development version
> instead, use `coatless-rpkg/checktor`, which installs from GitHub.

## Failing loudly, not silently

A red X that says only "exit code 1" is a riddle, not a report. When a build
fails, you want the diagnosis in the log, not a scavenger hunt. Run the full
`checktor()` once and print the report before you quit:

```r
results <- checktor::checktor()

if (!checktor::is_healthy(results)) {
  writeLines(checktor::health_report(results))  # the diagnosis, in the log
  quit(status = 1)                              # then fail the build
}
```

`health_report()` returns the findings as Markdown, so the failing log reads
like a chart at the foot of a hospital bed, showing what is wrong and where.
Point its `file =` argument at a path and you can just as easily upload the
report as a build artifact for the squeamish who prefer not to read CI logs.

## Tuning the examination

CI is chatty by nature, so quiet the doctor down. `configure_doctor()` sets the
verbosity and progress defaults once, at the top of a script, and every later
call inherits them:

```r
checktor::configure_doctor(verbose_default = FALSE, progress_default = FALSE)
```

You can also run a single category when that is all you care about, since each
`diagnose_*_issues()` function stands on its own, and a single check the same way
through its `lab_*()` function:

```r
# Gate only on DESCRIPTION-field problems
desc <- checktor::diagnose_description_issues(".", verbose = FALSE)
if (!checktor::is_healthy(desc)) quit(status = 1)

# Or on one check on its own
tf <- checktor::lab_tf_usage(".", verbose = FALSE)
if (!checktor::is_healthy(tf)) quit(status = 1)
```

A category or a single check has no verdict tiers to filter by, so `is_healthy()`
on one fails on any finding, opinion included. Use `checkup()` when the tiers
matter.

## Findings on the diff, not in the log

A log someone has to scroll is the weakest way to deliver a finding. Every forge
can put one next to the line that caused it, and `ci_report()` writes whichever
shape yours reads. With no arguments it examines the package, works out where it is
running, and emits the right thing.

```r
checktor::ci_report()
```

On GitHub Actions that prints workflow commands, and each finding appears on the
pull request diff:

```
::error file=R/plot.R,line=42,title=checktor%3A option_changes::Option changes check: plot.R:42
```

The colon in the title is written as `%3A`, the escape GitHub expects there. Gitea
and Forgejo Actions read the same commands, so the same call covers them.
`ci_report()` also recognises GitLab, Azure Pipelines and Jenkins from the
variables they set. Name the format when you want a different one, such as SARIF
for GitHub code scanning, or when the build runs somewhere it does not recognise:

| Forge | Format | What to do with it |
|---|---|---|
| GitHub, Gitea, Forgejo | `"github"` | Printed to the log, shown on the diff |
| GitLab | `"gitlab"` | Name it in `artifacts:reports:codequality:` |
| Azure Pipelines | `"azure"` | Printed to the log |
| Jenkins, reviewdog, review bots | `"checkstyle"` | Read the XML with the warnings plugin |
| GitHub code scanning | `"sarif"` | Upload with `github/codeql-action/upload-sarif` |

A GitLab job needs only this:

```yaml
checktor:
  image: rocker/r-ver:latest
  script:
    - Rscript -e 'install.packages("checktor"); checktor::ci_report(format = "gitlab")'
  artifacts:
    reports:
      codequality: gl-code-quality-report.json
```

`ci_report()` reports every tier, since an annotation is information rather than a
verdict. Keep `checkup()` for the verdict, so what fails a build stays a separate
decision from what gets pointed at. If you want an opinion to fail a build, that is
`checkup(severity = c("policy", "robustness", "opinion"))`, and the threshold stays
in one place rather than being spelled two ways that can disagree.

Checks that did not run are named once at the end, rather than one annotation each,
because a check that never happened should not compete for space with a finding:

```
::notice title=checktor%3A skipped::2 checks did not run: spelling, url_liveness
```

For the report formats that note goes to the job log instead, keeping the artifact a
clean document. Pass `skipped = FALSE` if you would rather not hear about it.

## Keeping the report

`health_report()` writes the whole consultation to a file, which an artifact step
can keep for later. Write the report in the step that runs the gate, before the
gate decides:

```yaml
      - name: Run extra-CRAN checks
        shell: Rscript {0}
        run: |
          results <- checktor::checktor(verbose = FALSE, progress = FALSE)
          invisible(checktor::health_report(results, file = "checktor-report.md"))
          if (!checktor::is_healthy(results)) quit(status = 1)

      - name: Upload checktor report
        if: always()
        uses: actions/upload-artifact@v7
        with:
          name: checktor-report
          path: checktor-report.md
```

Writing the report before the gate runs, and uploading it with `if: always()`,
means a failing build still leaves the findings behind after its log has scrolled
away. checktor's own workflow does exactly this.

## What a runner does not check

Two checks sit out in CI, because the result would say more about the runner than
about your package. The URL fetch waits until you are at the console, and spelling
waits until a spell-check backend is installed. Neither is reported as a pass. They
come back marked skipped, they are named in `metadata$skipped_checks`, `tidy()`
carries a `skipped` column, and `ci_report()` says so on the build itself, so a
green build never implies a check that never happened.

Turn the URL fetch on in a job that should do it, keeping in mind that a network
hiccup then becomes a red build:

```r
options(checktor.url_check = TRUE)
```

## Closer to the keyboard

CI is the backstop, not the first line. If you would rather hear about a problem
before it reaches a pull request, the same one-liner works as a local Git
pre-commit hook:

```bash
#!/usr/bin/env bash
# .git/hooks/pre-commit  (make it executable: chmod +x .git/hooks/pre-commit)
Rscript -e 'if (!checktor::checkup()) quit(status = 1)'
```

Now a commit that would have embarrassed you on CRAN never leaves your laptop.

It is the same call in both places, and the only thing that changes is how long
you wait to hear about it.

![The same `checkup()` call at three latencies. In a `.git/hooks/pre-commit` hook on your laptop it answers in seconds. In GitHub Actions on the pull request it answers in minutes. If you skip both, the same finding reaches you from a CRAN reviewer weeks later.](figures/ci-gates-light.svg){.ct-fig .ct-light}
![](figures/ci-gates-dark.svg){.ct-fig .ct-dark aria-hidden="true"}

## The takeaway

Wire `checkup()` into CI once and the rules stop living in your head, where they
were never safe anyway. The best submission is the one where the reviewer finds
nothing to say, and the surest way there is to have already said it to yourself,
automatically, on every push.

## See also

- [Getting Started with checktor](getting-started-with-checktor.html): the
  guided tour of the diagnostics.
- [Writing Your Own Checks](writing-checks.html): add project-specific checks
  against the parsed syntax tree.