andy/Programming-Contests
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

First version

Browse Source
This commit is contained in:
andy committed 2026-10-15 20:44:05 -03:00
commit 243e92a9dd
84 files changed
+4012
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

tests/README.md
+46
View File
@@ -0,0 +1,46 @@
# Testing a build
These tools start a throwaway copy of `build/acpc2026.qcow2` in QEMU and drive it from the build machine, over SSH and through the virtual keyboard, so that the image can be checked as a team and an administrator would use it. The built image itself is not modified. They need `qemu-storage-daemon`, `debugfs`, `sshpass` and Python with Pillow on the build machine, and the administrator password in `ADMIN_PASS`.
```
export ADMIN_PASS='PASSWORD'
tests/check_vm.sh start --wrong-clock --usb
tests/feature_check.sh
tests/technotes_check.sh
tests/ide_check.sh
tests/check_vm.sh stop
```
`feature_check.sh` runs, as root in the virtual machine, one check for each feature of the playbooks. `technotes_check.sh` runs, from the team account, the statements of the technical notes that can be checked on the machine. Both write their results to `test_report/` and end with the number of failed checks, which should be 0. When a playbook or the notes change, add the matching check.
| Tool | Use |
| --- | --- |
| `check_vm.sh start [--wrong-clock] [--usb]` | boots the copy (`tests/work/`) with SSH open for the tests and waits for the team desktop. `--wrong-clock` starts with the hardware clock 3 hours behind, `--usb` adds a USB controller |
| `check_vm.sh stop` | stops the virtual machine and deletes the copy |
| `vssh.sh team\|icpc 'command'` | runs a command over SSH |
| `gui.sh 'program arguments'` | starts a program on the team desktop |
| `key.sh 'text{{ret}}'` | types on the keyboard of the virtual machine. Special keys use the QEMU names, as in `{{esc}}`, `{{ctrl-s}}` and `{{f5}}`, and `key.sh @file` types a file |
| `shot.sh NAME` | saves a screenshot as `test_report/NAME.png` |
| `add_test_tools.sh` | installs `xdotool` in the copy only, for `click.sh` |
| `click.sh X Y [right\|double]` | clicks on the team desktop at a screen position, read on a screenshot (1280 by 800 pixels) |
| `usb.sh plug\|unplug` | plugs a small USB stick into a copy started with `--usb` |
| `make_report.py` | builds `test_report/index.html` from the screenshots, `test_report/captions.txt` (one `NAME \| caption` line per screenshot), `test_report/intro.txt` and the two checklists |
| `ide_check.sh` | verifies the installed IDE commands, C23/C++23, static linking, Java VM options, PyPy, paths with spaces, compiler probes and failed-build handling |
| `make_ide_report.py` | builds the separate, self-contained `test_report/ide-config/index.html` from reviewed screenshots and `results.json` |
A full test goes through the following steps, with a screenshot of each.
1. Boot with the wrong clock. The team desktop appears with the right time.
2. Open Firefox from the Applications menu, then the contest, standings, problems, login and status pages, and a few blocked sites.
3. Write a short program in each language in a different editor, then compile and run it in a terminal with the `compile` and `run` commands. `submit` prints the address of the problems on Kattis.
4. Run `/test_firewall.sh` in the team terminal.
5. Try to get around the restrictions from the team terminal (sudo, reboot, Wi-Fi, reading the administration scripts, a USB stick, a forced address, DNS).
6. Build and run a program in each language of each IDE, with the input typed in the IDE, and check which compiler or interpreter the IDE used. Cover all 19 combinations: Geany and VS Code and Eclipse (C, C++, Java, Python), Code::Blocks (C, C++), CLion (C, C++, Python), IntelliJ IDEA (Java), and PyCharm (Python). Use the actions in the technical notes; the ordinary Java run buttons in VS Code and Eclipse use ECJ, so test their supplied ACPC actions to verify javac. The fixtures in `ide_samples/` assert the language standards and runtime versions, exercise C++23 `std::println`, and check Java's seven VM options. Enter `3000000000 4`; each program must print `SUM = 3000000004` and exit successfully.
7. In the team terminal, run `su - icpc`, then `contest-restrictions status`, `off usb` (red banner), plug a USB stick, `report` and `on usb`. Check that the restrictions are active and the banner has disappeared.
8. Open **Contest Help (offline)** from the Contest menu. Disconnect the VM network with `python3 -I tests/monitor.py tests/work/monitor.sock 'set_link net0 off'`. Reload the page, search for `EOF`, follow an article link and open the local Technical Notes PDF. Use `key.sh` and `shot.sh` while SSH is unavailable. Restore the connection with `python3 -I tests/monitor.py tests/work/monitor.sock 'set_link net0 on'` before continuing other checks.
Do not run `/reset_team_home.sh` as part of this procedure or any automated test. It permanently erases team files and is reserved for an explicitly authorized manual staff action. The feature checks inspect its installation without executing it.
Keyboard input goes to the window that has the focus, so close dialogs and other windows before typing. First-launch windows depend on what was used before (PyCharm offers to import the settings of VS Code once VS Code has run), so the IDEs are worth testing on a fresh copy as well. The JetBrains IDEs take up to a minute to open in the virtual machine.
For the IDE report, record each reviewed result in `test_report/ide-config/results.json`: a `cases` list with `id`, `ide`, `language`, `action`, `status`, `evidence`, and `screenshots` (each with `file` and `caption`), plus `verdict`, `build`, `notes`, and `sources` (each with `title` and `url`). Keep the original report and screenshots. Run the final graphical checks on a fresh Ansible build without manually repairing compiler settings. Test fixtures may be copied into the VM; compile and run them through the actual IDE interface and inspect every selected screenshot.