Configuration

This page describes how to configure checker with .checker.yml, .deadlines.yml and .task.yml files.

You can refer to the course-template repository for examples of configuration files.

.checker.yml

This file describes how the checker will operate - how to export files, how to run pipelines and so on.

TBA

No json schema available yet, but you can refer to the checker.configs.checker.CheckerConfig in checker repository.
Or course-template repository.

![important]
The structure section requires glob patterns to be valid and will apply the same patterns recursively to all subdirectories.
The moment it faces .task.yml file, it will stop and use the parameters from this file recursively.
No ** patterns are allowed.

Example

The simple .checker.yml file is:

# .checker.yml
version: 1

# can be overwritten in .task.yml for individual tasks
structure:
  # ignore patterns: exclude from export, overwrite during testing
  ignore_patterns: [".git", ".idea", ".vscode", "__pycache__", ".venv", ".*_cache", "*.pyc"]
  # public patterns: include in export, overwrite during testing
  public_patterns: ["*", ".gitlab-ci-students.yml", ".gitignore"]
  # private patterns: exclude from export, overwrite during testing
  private_patterns: [".*"]

# default values for all tasks, can be overwritten in .task.yml params:
default_parameters:
  run_testing: true
  timeout: 10  # in seconds

# settings for export command, uses .deadlines.yml and `params` and each task params (in .task.yml)
export:
  destination: https://gitlab.manytask.org/test/public-test-repo

# settings for Tester, uses .checker.yml and `params` and each task params (in .task.yml)
testing:
  search_plugins: ["tools/plugins"]

  # run once per repo
  global_pipeline:
    - name: "Fail never"
      fail: never  # fast, after_all, never
      run: "run_script"
      args:
        origin: ${{ global.temp_dir }}
        script: "echo HI && false"

  # run once per task
  tasks_pipeline:
    - name: "Check forbidden regexps"
      fail: fast  # fast, after_all, never
      run: "check_regexps"
      args:
        origin: "${{ global.temp_dir }}/${{ task.task_sub_path }}"
        patterns: ["**/*.py"]
        regexps: ["exit(0)"]

    - name: "Run tests"
      run_if: ${{ parameters.run_testing }}
      fail: after_all  # fast, after_all, never
      run: "run_script"
      register_output: test_output
      args:
        origin: ${{ global.temp_dir }}
        script: "python -m pytest --tb=no -qq ${{ task.task_sub_path }}"
        timeout: ${{ parameters.timeout }}

  # will run once per task only if task_pipeline NOT failed
  report_pipeline:
    - name: "Report Score Manytask"
      run: "report_score_manytask"
      args:
        origin: "${{ global.temp_dir }}/${{ task.task_sub_path }}"
        patterns: ["**/*.py"]
        username: ${{ global.username }}
        task_name: ${{ task.task_name }}
        score: ${{ outputs.test_output.percentage }}

Templating in Tester Pipelines

You can use jinja2 templating in .checker.yml file pipeline arguments and run_if conditions.

The available variables are:

• global - global parameters ::: checker.tester.GlobalPipelineVariables handler: python
• task - task parameters ::: checker.tester.TaskPipelineVariables handler: python
• parameters - default parameters ::: checker.configs.checker.CheckerParametersConfig handler: python
• env - environment variables dict in the moment of running checker
• outputs - outputs of previous pipeline step if register_output is set, dict of string to checker.plugins.PluginOutput objects ::: checker.pipeline.PipelineOutputs handler: python

.deadlines.yml

This file describes deadlines for tasks. It is used by checker export command to export only tasks that are started.
Additionally, it is used by checker validate to ensure integrity of the deadlines and local files.

No json schema available yet, but you can refer to the checker.configs.deadlines.DeadlinesConfig in checker repository.
Or course-template repository.

Example

The simple .deadlines.yml file is:

# .deadlines.yml
version: 1

settings:
  timezone: Europe/Moscow

  deadlines: hard  # hard/interpolate
  max_submissions: 10  # optional
  submission_penalty: 0.1  # optional

  task_url: https://example.com/$GROUP_NAME/$TASK_NAME  # optional

schedule:
  - group: 1.FirstGroup
    enabled: true
    start: 2020-01-01 18:00:00
    steps:
      0.5: 7d
    end: 13d 03:00:00
    tasks:
      - task: hello_world
        score: 10
        bonus: 0
        special: 1
      - task: sum_a_b
        score: 5
        bonus: 5
        special: 0
      - task: disabled_task
        enabled: false
        score: 5

  - group: 2.SecondGroup
    start: 2020-02-01 18:00:00
    steps:
      0.9: 2020-02-08 18:00:00
      0.1: 14d
    tasks:
      - task: factorial
        score: 20

  - group: 3.ThirdGroup
    start: 2020-03-01 18:00:00
    tasks:
      - task: palindrome
        score: 0
        special: 2
        url: https://example.com

  - group: 4.FourthGroup
    enabled: false
    start: 2020-04-01 18:00:00
    tasks: []

.task.yml

This optional file describes task parameters. In used by checker when running tasks_pipeline to override default parameters, pipeline or layout (public or private files).

No json schema available yet, but you can refer to the checker.configs.task.TaskConfig in checker repository.
Or course-template repository.

Example

# .task.yml
version: 1

structure:  # optional
  ignore_patterns: ["*.pyc"]
  public_patterns: ["custom_public_file.txt"]
  private_patterns: [".*", "custom_private_tests.py"]

parameters:  # optional
  run_testing: true
  timeout: 10  # in seconds

task_pipeline:  # optional
  ...

report_pipeline:  # optional
  ...