First version
This commit is contained in:
andy committed
commit
243e92a9dd
84 files changed
+4012
No files matched your search
@@ -0,0 +1,46 @@
|
||||
# Testing a build
|
||||
|
||||
These tools start a throwaway copy of `build/acpc2026.qcow2` in QEMU and drive it from the build machine, over SSH and through the virtual keyboard, so that the image can be checked as a team and an administrator would use it. The built image itself is not modified. They need `qemu-storage-daemon`, `debugfs`, `sshpass` and Python with Pillow on the build machine, and the administrator password in `ADMIN_PASS`.
|
||||
|
||||
```
|
||||
export ADMIN_PASS='PASSWORD'
|
||||
tests/check_vm.sh start --wrong-clock --usb
|
||||
tests/feature_check.sh
|
||||
tests/technotes_check.sh
|
||||
tests/ide_check.sh
|
||||
tests/check_vm.sh stop
|
||||
```
|
||||
|
||||
`feature_check.sh` runs, as root in the virtual machine, one check for each feature of the playbooks. `technotes_check.sh` runs, from the team account, the statements of the technical notes that can be checked on the machine. Both write their results to `test_report/` and end with the number of failed checks, which should be 0. When a playbook or the notes change, add the matching check.
|
||||
|
||||
| Tool | Use |
|
||||
| --- | --- |
|
||||
| `check_vm.sh start [--wrong-clock] [--usb]` | boots the copy (`tests/work/`) with SSH open for the tests and waits for the team desktop. `--wrong-clock` starts with the hardware clock 3 hours behind, `--usb` adds a USB controller |
|
||||
| `check_vm.sh stop` | stops the virtual machine and deletes the copy |
|
||||
| `vssh.sh team\|icpc 'command'` | runs a command over SSH |
|
||||
| `gui.sh 'program arguments'` | starts a program on the team desktop |
|
||||
| `key.sh 'text{{ret}}'` | types on the keyboard of the virtual machine. Special keys use the QEMU names, as in `{{esc}}`, `{{ctrl-s}}` and `{{f5}}`, and `key.sh @file` types a file |
|
||||
| `shot.sh NAME` | saves a screenshot as `test_report/NAME.png` |
|
||||
| `add_test_tools.sh` | installs `xdotool` in the copy only, for `click.sh` |
|
||||
| `click.sh X Y [right\|double]` | clicks on the team desktop at a screen position, read on a screenshot (1280 by 800 pixels) |
|
||||
| `usb.sh plug\|unplug` | plugs a small USB stick into a copy started with `--usb` |
|
||||
| `make_report.py` | builds `test_report/index.html` from the screenshots, `test_report/captions.txt` (one `NAME \| caption` line per screenshot), `test_report/intro.txt` and the two checklists |
|
||||
| `ide_check.sh` | verifies the installed IDE commands, C23/C++23, static linking, Java VM options, PyPy, paths with spaces, compiler probes and failed-build handling |
|
||||
| `make_ide_report.py` | builds the separate, self-contained `test_report/ide-config/index.html` from reviewed screenshots and `results.json` |
|
||||
|
||||
A full test goes through the following steps, with a screenshot of each.
|
||||
|
||||
1. Boot with the wrong clock. The team desktop appears with the right time.
|
||||
2. Open Firefox from the Applications menu, then the contest, standings, problems, login and status pages, and a few blocked sites.
|
||||
3. Write a short program in each language in a different editor, then compile and run it in a terminal with the `compile` and `run` commands. `submit` prints the address of the problems on Kattis.
|
||||
4. Run `/test_firewall.sh` in the team terminal.
|
||||
5. Try to get around the restrictions from the team terminal (sudo, reboot, Wi-Fi, reading the administration scripts, a USB stick, a forced address, DNS).
|
||||
6. Build and run a program in each language of each IDE, with the input typed in the IDE, and check which compiler or interpreter the IDE used. Cover all 19 combinations: Geany and VS Code and Eclipse (C, C++, Java, Python), Code::Blocks (C, C++), CLion (C, C++, Python), IntelliJ IDEA (Java), and PyCharm (Python). Use the actions in the technical notes; the ordinary Java run buttons in VS Code and Eclipse use ECJ, so test their supplied ACPC actions to verify javac. The fixtures in `ide_samples/` assert the language standards and runtime versions, exercise C++23 `std::println`, and check Java's seven VM options. Enter `3000000000 4`; each program must print `SUM = 3000000004` and exit successfully.
|
||||
7. In the team terminal, run `su - icpc`, then `contest-restrictions status`, `off usb` (red banner), plug a USB stick, `report` and `on usb`. Check that the restrictions are active and the banner has disappeared.
|
||||
8. Open **Contest Help (offline)** from the Contest menu. Disconnect the VM network with `python3 -I tests/monitor.py tests/work/monitor.sock 'set_link net0 off'`. Reload the page, search for `EOF`, follow an article link and open the local Technical Notes PDF. Use `key.sh` and `shot.sh` while SSH is unavailable. Restore the connection with `python3 -I tests/monitor.py tests/work/monitor.sock 'set_link net0 on'` before continuing other checks.
|
||||
|
||||
Do not run `/reset_team_home.sh` as part of this procedure or any automated test. It permanently erases team files and is reserved for an explicitly authorized manual staff action. The feature checks inspect its installation without executing it.
|
||||
|
||||
Keyboard input goes to the window that has the focus, so close dialogs and other windows before typing. First-launch windows depend on what was used before (PyCharm offers to import the settings of VS Code once VS Code has run), so the IDEs are worth testing on a fresh copy as well. The JetBrains IDEs take up to a minute to open in the virtual machine.
|
||||
|
||||
For the IDE report, record each reviewed result in `test_report/ide-config/results.json`: a `cases` list with `id`, `ide`, `language`, `action`, `status`, `evidence`, and `screenshots` (each with `file` and `caption`), plus `verdict`, `build`, `notes`, and `sources` (each with `title` and `url`). Keep the original report and screenshots. Run the final graphical checks on a fresh Ansible build without manually repairing compiler settings. Test fixtures may be copied into the VM; compile and run them through the actual IDE interface and inspect every selected screenshot.
|
||||
Executable
+12
@@ -0,0 +1,12 @@
|
||||
#!/bin/bash
|
||||
# Installs xdotool in the throwaway test VM (not in the built image), for
|
||||
# mouse clicks with tests/click.sh. The firewall is turned off during the
|
||||
# installation and back on afterwards. ADMIN_PASS=... tests/add_test_tools.sh
|
||||
cd "$(dirname "$0")/.."
|
||||
tests/vssh.sh icpc "echo '${ADMIN_PASS:?set ADMIN_PASS}' | sudo -S -p '' bash -c '
|
||||
command -v xdotool >/dev/null && exit 0
|
||||
contest-restrictions off firewall >/dev/null
|
||||
apt-get install -y -qq xdotool >/dev/null 2>&1
|
||||
contest-restrictions on firewall >/dev/null
|
||||
/refresh_rules.sh
|
||||
command -v xdotool'"
|
||||
Executable
+90
@@ -0,0 +1,90 @@
|
||||
#!/bin/bash
|
||||
# Test virtual machine for the built image (build/acpc2026.qcow2).
|
||||
#
|
||||
# tests/check_vm.sh start [--wrong-clock] [--usb] boot a throwaway copy
|
||||
# tests/check_vm.sh stop power it off, delete the copy
|
||||
#
|
||||
# The copy (tests/work/check.qcow2) opens SSH for the tests: password login,
|
||||
# no DenyGroups, and the firewall accepts tcp 22. The built image itself is
|
||||
# not modified. --wrong-clock starts with the hardware clock 3 hours behind
|
||||
# (as on a lab machine running Windows), --usb adds a USB controller so that
|
||||
# USB devices can be plugged in with tests/usb.sh.
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
W=tests/work
|
||||
MON=$W/monitor.sock
|
||||
BUILT=$(realpath build/acpc2026.qcow2)
|
||||
running() { pgrep -f "^qemu-system-x86_64 -name acpc2026-check" >/dev/null; }
|
||||
|
||||
case "${1:-}" in
|
||||
start)
|
||||
shift
|
||||
running && { echo "The test VM is already running." >&2; exit 1; }
|
||||
mkdir -p "$W"
|
||||
qemu-img create -q -f qcow2 -b "$BUILT" -F qcow2 "$W/check.qcow2"
|
||||
for d in /usr/share/edk2/x64 /usr/share/OVMF /usr/share/edk2/ovmf; do
|
||||
for c in OVMF_CODE.4m.fd OVMF_CODE_4M.fd OVMF_CODE.fd; do
|
||||
[ -f "$d/$c" ] && { CODE=$d/$c; VARS=$d/${c/CODE/VARS}; break 2; }
|
||||
done
|
||||
done
|
||||
cp "$VARS" "$W/vars.fd"
|
||||
# Open SSH in the copy, offline
|
||||
touch "$W/fuse.raw"
|
||||
qemu-storage-daemon --blockdev driver=file,node-name=f,filename="$W/check.qcow2" \
|
||||
--blockdev driver=qcow2,node-name=q,file=f \
|
||||
--export type=fuse,id=e,node-name=q,mountpoint="$W/fuse.raw",writable=on > "$W/qsd.log" 2>&1 &
|
||||
for _ in $(seq 1 20); do mountpoint -q "$W/fuse.raw" && break; sleep 0.5; done
|
||||
start=$(sfdisk -J "$W/fuse.raw" | python3 -I -c 'import json,sys; p=json.load(sys.stdin)["partitiontable"]["partitions"]; print(max(p,key=lambda x:x["size"])["start"])')
|
||||
D="$W/fuse.raw?offset=$((start * 512))"
|
||||
printf 'PasswordAuthentication yes\n' > "$W/00-test.conf"
|
||||
w() { debugfs -w -R "$1" "$D" >/dev/null 2>&1 || true; }
|
||||
w "rm /etc/ssh/sshd_config.d/icpc.conf"
|
||||
w "write $W/00-test.conf /etc/ssh/sshd_config.d/00-test.conf"
|
||||
w "mkdir /etc/systemd/system/firewall.service.d"
|
||||
w "write tests/test-ssh.conf /etc/systemd/system/firewall.service.d/test-ssh.conf"
|
||||
for f in /etc/ssh/sshd_config.d/00-test.conf /etc/systemd/system/firewall.service.d/test-ssh.conf; do
|
||||
for x in "mode 0100644" "uid 0" "gid 0"; do w "set_inode_field $f $x"; done
|
||||
done
|
||||
for x in "mode 040755" "uid 0" "gid 0"; do w "set_inode_field /etc/systemd/system/firewall.service.d $x"; done
|
||||
fusermount3 -u "$W/fuse.raw"
|
||||
pkill -f "^qemu-storage-daemon .*$W/check.qcow2" || true
|
||||
sleep 1
|
||||
EXTRA=()
|
||||
for a in "$@"; do
|
||||
case "$a" in
|
||||
--wrong-clock) EXTRA+=(-rtc "base=$(date -u -d '-3 hours' +%Y-%m-%dT%H:%M:%S)") ;;
|
||||
--usb) EXTRA+=(-device qemu-xhci,id=xhci) ;;
|
||||
esac
|
||||
done
|
||||
setsid qemu-system-x86_64 -name acpc2026-check -machine q35,accel=kvm -cpu host -m 6144 -smp 4 \
|
||||
-drive if=pflash,format=raw,readonly=on,file="$CODE" \
|
||||
-drive if=pflash,format=raw,file="$W/vars.fd" \
|
||||
-drive id=disk0,file="$W/check.qcow2",format=qcow2,if=none \
|
||||
-device virtio-blk-pci,drive=disk0,bootindex=1 \
|
||||
-netdev user,id=net0,hostfwd=tcp:127.0.0.1:2222-:22 -device virtio-net-pci,netdev=net0 \
|
||||
-vga virtio -display none -monitor unix:"$MON",server,nowait \
|
||||
-serial file:"$W/serial.log" "${EXTRA[@]}" > "$W/qemu.log" 2>&1 &
|
||||
for _ in $(seq 1 120); do
|
||||
tests/vssh.sh team true 2>/dev/null && break
|
||||
sleep 3
|
||||
done
|
||||
# Wait for the desktop session of the team account
|
||||
for _ in $(seq 1 60); do
|
||||
tests/vssh.sh team 'loginctl list-sessions --no-legend' 2>/dev/null | grep -q seat0 && break
|
||||
sleep 2
|
||||
done
|
||||
echo "Test VM ready (SSH 127.0.0.1:2222, monitor $MON)."
|
||||
;;
|
||||
stop)
|
||||
if running; then
|
||||
python3 -I tests/monitor.py "$MON" quit || true
|
||||
for _ in $(seq 1 30); do running || break; sleep 1; done
|
||||
fi
|
||||
rm -rf "$W"
|
||||
echo "Test VM stopped."
|
||||
;;
|
||||
*)
|
||||
sed -n '2,12p' "$0" | sed 's/^# \{0,1\}//'
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
Executable
+11
@@ -0,0 +1,11 @@
|
||||
#!/bin/bash
|
||||
# Mouse on the team desktop (needs tests/add_test_tools.sh), screen pixels:
|
||||
# tests/click.sh X Y left click
|
||||
# tests/click.sh X Y right right click
|
||||
# tests/click.sh X Y double double click
|
||||
case "${3:-left}" in
|
||||
right) action="click 3" ;;
|
||||
double) action="click --repeat 2 --delay 120 1" ;;
|
||||
*) action="click 1" ;;
|
||||
esac
|
||||
exec "$(dirname "$0")/vssh.sh" team "DISPLAY=:0 XAUTHORITY=\$HOME/.Xauthority xdotool mousemove $1 $2 sleep 0.2 $action"
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
#!/bin/bash
|
||||
# Runs the checklist of every playbook feature in the test VM, as root:
|
||||
# ADMIN_PASS=... tests/feature_check.sh -> test_report/feature_check.txt
|
||||
cd "$(dirname "$0")/.."
|
||||
mkdir -p test_report
|
||||
sshpass -p "${ADMIN_PASS:?set ADMIN_PASS}" scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
|
||||
-o LogLevel=ERROR -P 2222 tests/feature_check_vm.sh icpc@127.0.0.1:/tmp/feature_check_vm.sh
|
||||
tests/vssh.sh icpc "echo '$ADMIN_PASS' | sudo -S -p '' bash /tmp/feature_check_vm.sh" | tee test_report/feature_check.txt
|
||||
Executable
+123
@@ -0,0 +1,123 @@
|
||||
#!/bin/bash
|
||||
# Runs inside the VM as root. Checks every feature of the Ansible playbooks.
|
||||
pass=0; fail=0
|
||||
c() { # c "description" command...
|
||||
local d=$1; shift
|
||||
if eval "$@" >/dev/null 2>&1; then echo "PASS $d"; pass=$((pass+1)); else echo "FAIL $d"; fail=$((fail+1)); fi
|
||||
}
|
||||
as_team() { runuser -u team -- bash -c "$1"; }
|
||||
section() { echo; echo "== $1"; }
|
||||
|
||||
section "accounts.yml / finish.yml"
|
||||
c "team is logged in automatically on the screen" "loginctl list-sessions --no-legend | grep -q 'team seat0'"
|
||||
c "team is not in the sudo group" "! id -nG team | grep -qw sudo"
|
||||
c "team is not in the lpadmin group" "! id -nG team | grep -qw lpadmin"
|
||||
c "icpc is in the sudo group" "id -nG icpc | grep -qw sudo"
|
||||
c "root password is locked" "passwd -S root | grep -q ' L '"
|
||||
c "sudo rule of vscode-extension-install removed" "[ ! -e /etc/sudoers.d/vscode-extension-install ]"
|
||||
c "vscode-extension-install not started at login" "grep -q '^Hidden=true' /etc/xdg/autostart/vscode-extension-install.desktop"
|
||||
c "VS Code extensions mounted for team at boot" "systemctl is-active -q team-vscode-extensions && findmnt /home/team/.vscode/extensions"
|
||||
c "boot menu: superuser and password" "grep -q 'set superusers=\"admin\"' /boot/grub/grub.cfg && grep -q password_pbkdf2 /boot/grub/grub.cfg"
|
||||
c "boot menu: normal entries unrestricted" "grep -q -- '--unrestricted' /boot/grub/grub.cfg"
|
||||
c "boot menu: no recovery entries" "! grep -q recovery /boot/grub/grub.cfg"
|
||||
c "boot menu password file is root only" "[ \$(stat -c %a /etc/grub.d/01_acpc_password) = 700 ]"
|
||||
c "SSH settings of the build removed" "[ ! -e /etc/ssh/sshd_config.d/00-build.conf ]"
|
||||
|
||||
section "languages.yml"
|
||||
c "gcc-15 15.2.0" "[ \$(gcc-15 -dumpfullversion) = 15.2.0 ]"
|
||||
c "g++-15 15.2.0" "[ \$(g++-15 -dumpfullversion) = 15.2.0 ]"
|
||||
c "java 25.0.4.1" "java -version 2>&1 | grep -q '\"25.0.4.1\"'"
|
||||
c "javac 25.0.4.1" "[ \"\$(javac -version 2>&1)\" = 'javac 25.0.4.1' ]"
|
||||
c "PyPy 7.3.23 / Python 3.11.15" "pypy3 -c 'import sys; assert sys.pypy_version_info[:3]==(7,3,23) and sys.version_info[:3]==(3,11,15)'"
|
||||
c "Java packages held at 25.0.4.1" "apt-mark showhold | grep -q openjdk-25-jdk-headless"
|
||||
c "compilec uses the Kattis command" "grep -q 'gcc-15 -g -O2 -o a.out -std=gnu23 -static \"\$@\" -lm' /usr/local/bin/compilec"
|
||||
c "compilecpp uses the Kattis command" "grep -q 'g++-15 -g -O2 -o a.out -std=gnu++23 -static -lrt -Wl,--whole-archive -lpthread -Wl,--no-whole-archive' /usr/local/bin/compilecpp"
|
||||
c "compilejava uses the Kattis command" "grep -q \"javac -encoding UTF-8 -sourcepath . -cp './\\*' -d .\" /usr/local/bin/compilejava"
|
||||
c "runjava uses the Kattis options" "grep -q 'UseSerialGC -Xss64m -XX:TieredStopAtLevel=1 -XX:ActiveProcessorCount=2' /usr/local/bin/runjava"
|
||||
c "runpython3 uses pypy3" "grep -q 'exec pypy3' /usr/local/bin/runpython3"
|
||||
c "compilegcc and compileg++ are other names" "[ \$(readlink /usr/local/bin/compilegcc) = /usr/local/bin/compilec ] && [ \$(readlink /usr/local/bin/compileg++) = /usr/local/bin/compilecpp ]"
|
||||
c "technical notes are the ACPC 2026 ones" "pdftotext /usr/share/doc/icpc/TechNotes.pdf - | grep -q 'Atlantic Canadian Programming Competition 2026'"
|
||||
|
||||
section "firefox.yml"
|
||||
P=/usr/lib/firefox/distribution/policies.json
|
||||
c "home page is the contest" "python3 -c \"import json;p=json.load(open('$P'))['policies'];assert p['Homepage']['URL']=='https://atlantic-canada26.kattis.com' and p['Homepage']['Locked']\""
|
||||
c "contest, offline help and documentation bookmarks" "python3 -c \"import json;p=json.load(open('$P'))['policies'];assert [b['Title'] for b in p['Bookmarks']]==['Contest','Problems','Standings','Contest Help (offline)','Language Documentation','Technical Notes'];assert p['Bookmarks'][3]['URL']=='file:///usr/share/doc/acpc/help/index.html'\""
|
||||
c "encrypted client hello off (locked)" "python3 -c \"import json;p=json.load(open('$P'))['policies']['Preferences'];assert p['network.dns.echconfig.enabled']=={'Value':False,'Status':'locked'}\""
|
||||
c "extension installs blocked" "python3 -c \"import json;p=json.load(open('$P'))['policies'];assert p['ExtensionSettings']['*']['installation_mode']=='blocked'\""
|
||||
c "about:config blocked" "python3 -c \"import json;p=json.load(open('$P'))['policies'];assert p['BlockAboutConfig'] is True\""
|
||||
|
||||
section "desktop.yml"
|
||||
c "World Finals menu entries removed" "! ls /usr/share/applications/{ccs,scoreboard,problems,ccsdocs,judgingnotes}.desktop"
|
||||
c "Contest menu: Contest, Problems, Standings and offline help" "ls /usr/share/applications/acpc-{contest,problems,standings,help}.desktop"
|
||||
c "Documentation menu entry named Language Documentation" "grep -q '^Name=Language Documentation' /usr/share/applications/devdocs.desktop"
|
||||
c "submit points to Kattis" "as_team 'submit' | grep -q 'This contest runs on Kattis'"
|
||||
c "pc2submit points to Kattis" "as_team 'pc2submit' | grep -q 'This contest runs on Kattis'"
|
||||
c "Kotlin compile/run commands removed" "[ ! -e /usr/local/bin/compilekotlin ] && [ ! -e /usr/local/bin/runkotlin ]"
|
||||
c "JetBrains agreement answered for team" "grep -rq eua_accepted_version /home/team/.java/.userPrefs/jetbrains/"
|
||||
c "JetBrains data sharing declined for team" "grep -q 'rsch.send.usage.stat:1.1:0' /home/team/.local/share/JetBrains/consentOptions/accepted"
|
||||
c "same answers in /etc/skel" "grep -rq eua_accepted_version /etc/skel/.java/.userPrefs/jetbrains/ && [ -f /etc/skel/.local/share/JetBrains/consentOptions/accepted ]"
|
||||
c "PyCharm: PyPy declared and used by default" "grep -q '/usr/local/bin/pypy3' /home/team/.config/JetBrains/PyCharm*/options/jdk.table.xml && grep -q 'PyPy 3.11' /home/team/.config/JetBrains/PyCharm*/options/project.default.xml"
|
||||
c "CLion, IntelliJ IDEA, PyCharm trust /home/team" "[ \$(grep -l '/home/team' /home/team/.config/JetBrains/*/options/trusted-paths.xml | wc -l) = 3 ]"
|
||||
c "same JetBrains settings in /etc/skel" "[ \$(ls /etc/skel/.config/JetBrains/*/options/trusted-paths.xml | wc -l) = 3 ]"
|
||||
c "Eclipse runs on Java 25.0.4.1" "grep -qx '/usr/lib/jvm/java-25-openjdk-amd64/bin/java' /opt/eclipse/eclipse.ini"
|
||||
c "Geany C build uses the contest compiler driver" "grep -q '^FT_01_CM=acpc-gcc' /usr/share/geany/filedefs/filetypes.c"
|
||||
c "Geany C++ build uses the contest compiler driver" "grep -q '^FT_01_CM=acpc-g++' /usr/share/geany/filedefs/filetypes.cpp"
|
||||
c "Geany runs Python with PyPy" "grep -q '^EX_00_CM=pypy3' /usr/share/geany/filedefs/filetypes.python"
|
||||
c "VS Code uses PyPy for Python" "grep -q '/usr/local/bin/pypy3' /home/team/.config/Code/User/settings.json"
|
||||
|
||||
c "Geany Java uses contest compilation and VM options" "grep -q '^FT_01_CM=compilejava' /usr/share/geany/filedefs/filetypes.java && grep -q '^EX_00_CM=runjava' /usr/share/geany/filedefs/filetypes.java"
|
||||
c "CLion C and C++ toolchain drivers" "grep -q 'customCCompilerPath' /home/team/.config/JetBrains/CLion*/options/linux/toolchains.xml && grep -q 'customCXXCompilerPath' /home/team/.config/JetBrains/CLion*/options/linux/toolchains.xml"
|
||||
c "CLion Python defaults to PyPy" "grep -q /usr/local/bin/pypy3 /home/team/.config/JetBrains/CLion*/options/jdk.table.xml"
|
||||
c "IntelliJ Java SDK and VM options" "grep -q java-25-openjdk /home/team/.config/JetBrains/IntelliJIdea*/options/jdk.table.xml && grep -q UseSerialGC /home/team/.config/JetBrains/IntelliJIdea*/options/project.default.xml"
|
||||
c "CodeBlocks C and C++ compiler profile" "grep -q acpc-gcc /home/team/.config/codeblocks/default.conf && grep -q acpc-g++ /home/team/.config/codeblocks/default.conf"
|
||||
c "VS Code contest build actions and F6 shortcut" "grep -q acpc-solution /home/team/.config/Code/User/tasks.json && grep -q f6 /home/team/.config/Code/User/keybindings.json"
|
||||
c "Eclipse menu initializes the contest workspace" "grep -q '^Exec=/usr/local/bin/acpc-eclipse' /usr/share/applications/Eclipse.desktop"
|
||||
c "Eclipse PyDev uses PyPy without changing Java/CDT build preferences" "grep -q /usr/local/bin/pypy3 /home/team/eclipse-workspace/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.python.pydev.prefs && grep -qx CHECK_PREFERRED_PYDEV_SETTINGS=false /home/team/eclipse-workspace/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.python.pydev.prefs"
|
||||
c "IDE profiles restored by reset from skel" "test -f /etc/skel/.config/codeblocks/default.conf && test -f /etc/skel/.config/Code/User/tasks.json && test -f /etc/skel/eclipse-workspace/.metadata/.plugins/org.eclipse.debug.core/.launches/ACPC\ Build\ and\ run.launch"
|
||||
|
||||
section "docs.yml"
|
||||
c "offline help is installed, root-owned and readable by team" "[ \"\$(stat -c '%U:%G:%a' /usr/share/doc/acpc/help/index.html)\" = root:root:644 ] && as_team 'test -r /usr/share/doc/acpc/help/index.html'"
|
||||
c "offline help menu opens the local document" "grep -qx 'Exec=firefox file:///usr/share/doc/acpc/help/index.html' /usr/share/applications/acpc-help.desktop"
|
||||
c "DevDocs runs" "systemctl is-active -q container-devdocs && curl -sk -o /dev/null https://devdocs/"
|
||||
c "DevDocs has exactly C, C++, OpenJDK 25, Python 3.11" "[ \"\$(podman exec devdocs ls public/docs | tr '\n' ' ')\" = 'c cpp docs.json openjdk~25 python~3.11 ' ]"
|
||||
c "Kotlin gone from the compiled list" "! podman exec devdocs sh -c 'grep -q kotlin public/assets/docs-*.js'"
|
||||
c "no 'must be loaded from devdocs.io' notice" "podman exec devdocs sh -c 'grep -q production_host:\\\"devdocs\\\" public/assets/application-*.js'"
|
||||
|
||||
section "firewall.yml"
|
||||
c "firewall, proxy and DNS filter running" "systemctl is-active -q firewall contest-proxy contest-dns"
|
||||
c "test_firewall.sh: 15 OK, no problem (as team)" "[ \$(as_team /test_firewall.sh | grep -c '^OK') = 15 ]"
|
||||
c "web egress only for the proxy account" "nft list chain inet filter output | grep -q \"skuid \$(id -u contest-proxy) tcp dport 443 accept\""
|
||||
c "DNS egress only for the proxy account" "nft list chain inet filter output | grep -q \"skuid \$(id -u contest-proxy) ip daddr 1.1.1.1 udp dport 53\""
|
||||
c "NTP egress only for chrony" "nft list chain inet filter output | grep -q \"skuid \$(id -u _chrony) ip daddr @ntp_ips udp dport 123 accept\""
|
||||
c "incoming connections dropped (policy drop)" "nft list chain inet filter input | grep -q 'policy drop'"
|
||||
c "resolv.conf points to the DNS filter" "[ \"\$(cat /etc/resolv.conf)\" = 'nameserver 127.0.0.1' ]"
|
||||
c "open.kattis.com in /etc/hosts" "grep -q 'open.kattis.com' /etc/hosts"
|
||||
c "clock synchronized by chrony" "chronyc -n sources | grep -q '^\\^\\*'"
|
||||
c "chrony does not write the hardware clock" "! grep -q '^rtcsync' /etc/chrony/chrony.conf"
|
||||
c "Ubuntu NTS sources disabled" "[ ! -e /etc/chrony/sources.d/ubuntu-ntp-pools.sources ]"
|
||||
c "clock fallback timer enabled" "systemctl is-enabled -q time_from_contest.timer"
|
||||
c "time zone America/Moncton" "[ \$(readlink -f /etc/localtime) = /usr/share/zoneinfo/America/Moncton ]"
|
||||
|
||||
section "lockdown.yml"
|
||||
c "no local AI models in the JetBrains IDEs" "[ -z \"\$(find /opt -name 'full-line-model-*' -o -name 'full-line-native-server-*')\" ]"
|
||||
c "USBGuard running, rejects new devices" "systemctl is-active -q usbguard && grep -q '^ImplicitPolicyTarget=reject' /etc/usbguard/usbguard-daemon.conf && grep -q '^PresentDevicePolicy=allow' /etc/usbguard/usbguard-daemon.conf"
|
||||
c "polkit: removable media refused to teams" "[ -f /etc/polkit-1/rules.d/10-acpc-no-removable-media.rules ]"
|
||||
c "polkit: network changes refused to teams" "[ -f /etc/polkit-1/rules.d/10-acpc-no-network-changes.rules ]"
|
||||
c "polkit: restart/power off refused to teams" "[ -f /etc/polkit-1/rules.d/10-acpc-no-power-actions.rules ]"
|
||||
c "udev: no direct access to phones" "[ -f /etc/udev/rules.d/72-acpc-no-phone-access.rules ]"
|
||||
c "team cannot reboot" "! systemd-run --machine=team@ --user --wait --pipe --quiet systemctl reboot"
|
||||
c "team cannot change the network" "systemd-run --machine=team@ --user --wait --pipe --quiet nmcli general permissions | grep -q 'network-control *no'"
|
||||
c "Ctrl+Alt+Del masked" "[ \$(systemctl is-enabled ctrl-alt-del.target 2>&1) = masked ]"
|
||||
c "SysRq off" "[ \$(cat /proc/sys/kernel/sysrq) = 0 ]"
|
||||
c "power/reboot keys ignored" "grep -q '^HandlePowerKey=ignore' /etc/systemd/logind.conf.d/90-acpc-keys.conf"
|
||||
c "desktop waits for the clock" "systemctl is-enabled -q wait-for-time"
|
||||
c "automatic updates masked" "[ \"\$(systemctl is-enabled apt-daily.timer apt-daily-upgrade.timer unattended-upgrades 2>&1 | sort -u)\" = masked ]"
|
||||
c "contest-restrictions installed, root only" "[ \$(stat -c %a /usr/local/sbin/contest-restrictions) = 700 ]"
|
||||
c "contest-restrictions status: all on" "[ \$(contest-restrictions status | grep -c ' on\$') = 3 ]"
|
||||
c "banner service running (root)" "systemctl is-active -q contest-restrictions-warning"
|
||||
c "/etc/contest-restrictions root only" "[ \$(stat -c %a /etc/contest-restrictions) = 700 ]"
|
||||
c "admin scripts unreadable by team" "! as_team 'cat /disable_firewall.sh /usr/local/sbin/contest-restrictions /reset_team_home.sh'"
|
||||
c "reset script installed" "[ -x /reset_team_home.sh ]"
|
||||
|
||||
echo
|
||||
echo "$pass passed, $fail failed"
|
||||
Executable
+3
@@ -0,0 +1,3 @@
|
||||
#!/bin/bash
|
||||
# Starts a program on the desktop of the team account: tests/gui.sh 'firefox https://devdocs/'
|
||||
exec "$(dirname "$0")/vssh.sh" team "export DISPLAY=:0 XAUTHORITY=\$HOME/.Xauthority; (setsid $1 >/dev/null 2>&1 &)"
|
||||
Executable
+7
@@ -0,0 +1,7 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
mkdir -p test_report/ide-config
|
||||
tar -C tests -cf - ide_check_vm.py ide_samples | tests/vssh.sh team \
|
||||
'mkdir -p /tmp/acpc-ide-check && tar -xf - -C /tmp/acpc-ide-check'
|
||||
tests/vssh.sh team 'python3 /tmp/acpc-ide-check/ide_check_vm.py' | tee test_report/ide-config/integration-check.txt
|
||||
Executable
+129
@@ -0,0 +1,129 @@
|
||||
#!/usr/bin/python3
|
||||
"""Integration checks in the built image; graphical checks are recorded separately."""
|
||||
import configparser
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import xml.etree.ElementTree as ET
|
||||
|
||||
passed = failed = 0
|
||||
|
||||
def check(name, fn):
|
||||
global passed, failed
|
||||
try:
|
||||
fn()
|
||||
print('PASS ', name, flush=True)
|
||||
passed += 1
|
||||
except Exception as error:
|
||||
print('FAIL ', name, ':', error, flush=True)
|
||||
failed += 1
|
||||
|
||||
def require(condition, detail='assertion failed'):
|
||||
if not condition:
|
||||
raise AssertionError(detail)
|
||||
|
||||
def run(args, **kwargs):
|
||||
result = subprocess.run(args, text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=45, **kwargs)
|
||||
require(result.returncode == 0, result.stdout)
|
||||
return result.stdout
|
||||
|
||||
samples = Path(__file__).parent / 'ide_samples'
|
||||
with tempfile.TemporaryDirectory(prefix='ACPC IDE check ') as directory:
|
||||
work = Path(directory)
|
||||
for filename in ('sum.c', 'sum.cpp', 'Sum.java', 'sum.py'):
|
||||
case = work / filename.replace('.', '-')
|
||||
case.mkdir()
|
||||
source = case / filename
|
||||
shutil.copy(samples / filename, source)
|
||||
def exercise(source=source):
|
||||
output = run(['acpc-solution', 'run', str(source)], input='3000000000 4\n')
|
||||
require('SUM = 3000000004' in output, output)
|
||||
if source.suffix in ('.c', '.cpp'):
|
||||
require('statically linked' in run(['file', str(source.with_suffix(''))]))
|
||||
if source.suffix == '.java':
|
||||
require(int.from_bytes(source.with_suffix('.class').read_bytes()[6:8], 'big') == 69)
|
||||
print(' ' + output.strip().replace('\n', '\n '))
|
||||
check('build and execute ' + filename + ' (path contains spaces)', exercise)
|
||||
|
||||
def driver_flags():
|
||||
log = work / 'compiler.jsonl'
|
||||
for lang, driver, old_standard, expected, library in (
|
||||
('c', 'acpc-gcc', 'c99', 'gnu23', '-lm'),
|
||||
('cpp', 'acpc-g++', 'c++17', 'gnu++23', '-lpthread'),
|
||||
):
|
||||
source = work / ('sum-' + lang) / ('sum.' + lang)
|
||||
run([driver, '-O0', '-std=' + old_standard, str(source), '-o', str(work / driver)],
|
||||
env={**os.environ, 'ACPC_COMPILER_LOG': str(log)})
|
||||
args = json.loads(log.read_text().splitlines()[-1])['argv']
|
||||
require('-O0' not in args and '-O2' in args and '-std=' + expected in args, str(args))
|
||||
require('-static' in args and library in args, str(args))
|
||||
require('15.2.0' in run(['acpc-gcc', '-dumpfullversion']))
|
||||
require('15.2.0' in run(['acpc-g++', '--version']))
|
||||
output = run(['acpc-cc', '-x', 'c++', '-dM', '-E', '-'], input='')
|
||||
require('#define __cplusplus 202302L' in output, 'C++ compiler probes must use the C++ backend')
|
||||
response = work / 'compiler probe.args'
|
||||
response.write_text('-x c++ -std=c++17 -dM -E -')
|
||||
output = run(['acpc-cc', '@' + str(response)], input='')
|
||||
require('#define __cplusplus 202302L' in output,
|
||||
'CLion response-file probes must see the same C++23 standard as compilation')
|
||||
check('compiler drivers override generic project flags, link statically and answer IDE probes', driver_flags)
|
||||
|
||||
def failed_build():
|
||||
case = work / 'sum-cpp'
|
||||
source = case / 'sum.cpp'
|
||||
source.write_text('#error deliberate bad source\n')
|
||||
result = subprocess.run(['acpc-solution', 'run', str(source)], text=True, input='1 2\n', capture_output=True)
|
||||
require(result.returncode != 0 and 'SUM =' not in result.stdout and 'Run:' not in result.stdout,
|
||||
'a failed build must never execute the previous binary')
|
||||
check('failed build does not execute a stale binary', failed_build)
|
||||
|
||||
def geany_commands():
|
||||
for language, filename in [('c', 'sum.c'), ('cpp', 'sum.cpp'), ('java', 'Sum.java'), ('python', 'sum.py')]:
|
||||
conf = configparser.ConfigParser(interpolation=None, strict=False)
|
||||
conf.read('/usr/share/geany/filedefs/filetypes.' + language)
|
||||
source = work / ('geany ' + language) / filename
|
||||
source.parent.mkdir()
|
||||
shutil.copy(samples / filename, source)
|
||||
for key in ('FT_00_CM', 'FT_01_CM', 'EX_00_CM'):
|
||||
command = conf['build-menu'][key].replace('%f', source.name).replace('%e', source.stem)
|
||||
result = run(['bash', '-c', command], cwd=source.parent, input='3000000000 4\n')
|
||||
if key == 'EX_00_CM': require('SUM = 3000000004' in result, result)
|
||||
if key == 'FT_00_CM' and source.suffix in ('.c', '.cpp'):
|
||||
require('relocatable' in run(['file', str(source.with_suffix('.o'))]))
|
||||
check('all four installed Geany build and execute commands work', geany_commands)
|
||||
|
||||
for home in (Path('/home/team'), Path('/etc/skel')):
|
||||
def profiles(home=home):
|
||||
cb = ET.parse(home / '.config/codeblocks/default.conf')
|
||||
cb_root = Path(cb.findtext('./compiler/sets/gcc/MASTER_PATH/str')) / 'bin'
|
||||
require((cb_root / cb.findtext('./compiler/sets/gcc/C_COMPILER/str')).resolve() == Path('/usr/local/bin/acpc-cc'))
|
||||
require((cb_root / cb.findtext('./compiler/sets/gcc/CPP_COMPILER/str')).resolve() == Path('/usr/local/bin/acpc-cc'))
|
||||
require(not Path(cb.findtext('./compiler/sets/gcc/C_COMPILER/str')).is_absolute(),
|
||||
'Code::Blocks prefixes MASTER_PATH/bin, including for an absolute filename')
|
||||
code = home / '.config/Code/User'
|
||||
tasks = json.loads((code / 'tasks.json').read_text())['tasks']
|
||||
require(any(t['label'] == 'ACPC: Build and run' and t['command'] == '/usr/local/bin/acpc-solution' for t in tasks))
|
||||
require(any(b.get('key') == 'f6' and b.get('args') == 'ACPC: Build and run' for b in json.loads((code / 'keybindings.json').read_text())))
|
||||
for product in ('CLion', 'PyCharm'):
|
||||
options = next((home / '.config/JetBrains').glob(product + '*/options'))
|
||||
require('/usr/local/bin/pypy3' in (options / 'jdk.table.xml').read_text())
|
||||
options = next((home / '.config/JetBrains').glob('IntelliJIdea*/options'))
|
||||
require('ACPC Java 25' in (options / 'jdk.table.xml').read_text())
|
||||
require('-XX:+UseSerialGC' in (options / 'project.default.xml').read_text())
|
||||
options = next((home / '.config/JetBrains').glob('CLion*/options'))
|
||||
chain = ET.parse(options / 'linux/toolchains.xml').find('./component/toolchains/toolchain')
|
||||
require(chain.get('customCCompilerPath') == '/usr/local/bin/acpc-gcc')
|
||||
require(chain.get('customCXXCompilerPath') == '/usr/local/bin/acpc-g++')
|
||||
workspace = home / 'eclipse-workspace/.metadata/.plugins'
|
||||
launch = ET.parse(workspace / 'org.eclipse.debug.core/.launches/ACPC Build and run.launch')
|
||||
require(any(n.get('value') == '/usr/local/bin/acpc-solution' for n in launch.getroot()))
|
||||
prefs = workspace / 'org.eclipse.core.runtime/.settings'
|
||||
require('/usr/local/bin/pypy3' in (prefs / 'org.python.pydev.prefs').read_text())
|
||||
require('java-25-openjdk' in (prefs / 'org.eclipse.jdt.launching.prefs').read_text())
|
||||
check(str(home) + ': settings for every IDE are present and parseable', profiles)
|
||||
|
||||
print(f'\n{passed} passed, {failed} failed')
|
||||
raise SystemExit(bool(failed))
|
||||
@@ -0,0 +1,16 @@
|
||||
import java.lang.management.ManagementFactory;
|
||||
import java.util.Scanner;
|
||||
public class Sum {
|
||||
public static void main(String[] args) {
|
||||
if (!System.getProperty("java.version").equals("25.0.4.1"))
|
||||
throw new IllegalStateException("Java 25.0.4.1 is required");
|
||||
var vmArgs = ManagementFactory.getRuntimeMXBean().getInputArguments();
|
||||
for (String flag : new String[]{"-Dfile.encoding=UTF-8", "-XX:+UseSerialGC", "-Xss64m", "-XX:TieredStopAtLevel=1", "-XX:ActiveProcessorCount=2", "-Xms2048m", "-Xmx2048m"})
|
||||
if (!vmArgs.contains(flag)) throw new IllegalStateException("Missing " + flag);
|
||||
System.out.println("Java " + System.getProperty("java.version"));
|
||||
System.out.println("VM: " + vmArgs);
|
||||
System.out.print("Input two integers: ");
|
||||
Scanner in = new Scanner(System.in);
|
||||
System.out.println("SUM = " + (in.nextLong() + in.nextLong()));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
#include <stdio.h>
|
||||
#ifdef __cplusplus
|
||||
#error This C file must be compiled as C, not C++.
|
||||
#endif
|
||||
#if __STDC_VERSION__ < 202311L
|
||||
#error C23 is required.
|
||||
#endif
|
||||
int main(void) {
|
||||
long long a, b;
|
||||
printf("C23 | GCC %s\nInput two integers: ", __VERSION__);
|
||||
fflush(stdout);
|
||||
if (scanf("%lld %lld", &a, &b) != 2) return 1;
|
||||
printf("SUM = %lld\n", a + b);
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
#include <iostream>
|
||||
#include <print>
|
||||
#if __cplusplus < 202302L
|
||||
#error C++23 is required.
|
||||
#endif
|
||||
int main() {
|
||||
std::println("C++23 | GCC {}", __VERSION__);
|
||||
std::cout << "Input two integers: " << std::flush;
|
||||
long long a, b;
|
||||
if (!(std::cin >> a >> b)) return 1;
|
||||
std::println("SUM = {}", a + b);
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
import platform
|
||||
import sys
|
||||
assert platform.python_implementation() == 'PyPy'
|
||||
assert sys.pypy_version_info[:3] == (7, 3, 23)
|
||||
assert sys.version_info[:3] == (3, 11, 15)
|
||||
print('PyPy 7.3.23 | Python ' + platform.python_version())
|
||||
values = input('Input two integers: ').split()
|
||||
print('SUM =', sum(map(int, values)))
|
||||
Executable
+4
@@ -0,0 +1,4 @@
|
||||
#!/bin/bash
|
||||
# Types on the keyboard of the test VM: tests/key.sh 'text{{ret}}' or tests/key.sh @file
|
||||
cd "$(dirname "$0")/.."
|
||||
exec python3 -I tests/type.py tests/work/monitor.sock "$1"
|
||||
@@ -0,0 +1,67 @@
|
||||
#!/usr/bin/python3
|
||||
"""Create a portable HTML report from reviewed screenshots and recorded results."""
|
||||
import base64
|
||||
from datetime import datetime, timezone
|
||||
import html
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
root = Path(__file__).resolve().parent.parent
|
||||
report = root / 'test_report/ide-config'
|
||||
data = json.loads((report / 'results.json').read_text())
|
||||
e = html.escape
|
||||
|
||||
def photo(name, caption):
|
||||
content = base64.b64encode((report / name).read_bytes()).decode()
|
||||
url = 'data:image/png;base64,' + content
|
||||
return f'<figure><button type="button" class="enlarge" aria-label="Enlarge {e(caption)}"><img src="{url}" loading="lazy" alt="{e(caption)}"></button><figcaption>{e(caption)} · <a href="{url}" download="{e(name)}">PNG</a></figcaption></figure>'
|
||||
|
||||
cases = data['cases']
|
||||
total = len(cases)
|
||||
passed = sum(c['status'] == 'pass' for c in cases)
|
||||
rows = []
|
||||
cards = []
|
||||
for case in cases:
|
||||
ident = case['id']
|
||||
rows.append(f'<tr><td><a href="#{e(ident)}">{e(case["ide"])}</a></td><td>{e(case["language"])}</td><td class="{e(case["status"])}">{e(case["status"].upper())}</td><td>{e(case["action"])}</td></tr>')
|
||||
pictures = ''.join(photo(p['file'], p['caption']) for p in case['screenshots'])
|
||||
cards.append(f'<article id="{e(ident)}" data-ide="{e(case["ide"])}"><h2>{e(case["ide"])} · {e(case["language"])} <span class="badge {e(case["status"])}">{e(case["status"].upper())}</span></h2><p><strong>Action:</strong> {e(case["action"])}</p><p>{e(case["evidence"])}</p>{pictures}</article>')
|
||||
|
||||
logs = []
|
||||
for name, title in [('integration-check.txt', 'Compiler and configuration integration checks'),
|
||||
('feature-check.txt', 'Image feature checks'), ('technotes-check.txt', 'Technical notes checks'),
|
||||
('idempotence.txt', 'Configuration reapplied without changes'),
|
||||
('reset-check.txt', 'Team-home reset and restored configuration'),
|
||||
('reset-integration-check.txt', 'Compiler integration checks after the team-home reset'),
|
||||
('build-summary.txt', 'Final image build and installed file checksums'),
|
||||
('gui-artifacts.txt', 'Artifacts created by graphical runs')]:
|
||||
path = report / name
|
||||
if path.exists():
|
||||
logs.append(f'<details><summary>{e(title)}</summary><pre>{e(path.read_text())}</pre></details>')
|
||||
sources = ''.join(f'<li><a href="{e(s["url"])}">{e(s["title"])}</a></li>' for s in data['sources'])
|
||||
fixtures = ''.join(f'<details><summary>{e(p.name)} — test source</summary><pre>{e(p.read_text())}</pre></details>' for p in sorted((root / 'tests/ide_samples').iterdir()) if p.is_file())
|
||||
options = ''.join(f'<option>{e(ide)}</option>' for ide in dict.fromkeys(c['ide'] for c in cases))
|
||||
notes = ''.join(f'<li>{e(note)}</li>' for note in data['notes'])
|
||||
build = e(data['build'])
|
||||
notes_page = report / 'technotes-final-3.png'
|
||||
instructions = ('<section class="panel"><h2>Instructions supplied to teams</h2><p>Page 3 of the updated TechNotes explains the tested action for every IDE and the first-launch dialogs. The PDF installed in the final image was checked against this copy.</p>' + photo(notes_page.name, 'TechNotes · IDE instructions, page 3') + '</section>') if notes_page.exists() else ''
|
||||
page = f'''<!doctype html>
|
||||
<html lang="en"><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>ACPC 2026 — IDE compiler verification</title>
|
||||
<style>
|
||||
:root{{font-family:system-ui,sans-serif;color:#172434;background:#f1f4f7;line-height:1.55}}body{{margin:0}}main{{max-width:1100px;margin:auto;padding:32px 22px 70px}}header,article,.panel{{background:white;border:1px solid #d7e0e8;border-radius:12px;padding:24px;margin:0 0 24px}}h1{{font-size:2rem;line-height:1.2;margin:.2em 0 .6em}}h2{{font-size:1.35rem}}.eyebrow{{color:#526579;font-weight:650;letter-spacing:.07em;text-transform:uppercase;font-size:.8rem}}a{{color:#155b96}}table{{width:100%;border-collapse:collapse}}th,td{{text-align:left;border-bottom:1px solid #dce4ec;padding:9px 8px}}th{{background:#edf3f8}}.pass{{color:#11603d}}.fail{{color:#a12121}}.pending{{color:#835800}}.badge{{border:1px solid currentColor;border-radius:5px;padding:2px 7px;font-size:.75rem;vertical-align:middle;margin-left:8px}}img{{max-width:100%;height:auto;display:block;border:1px solid #ced9e3;border-radius:6px}}figure{{margin:20px 0 0}}figcaption{{font-size:.9rem;margin:7px 0;color:#42576a}}pre{{white-space:pre-wrap;overflow-wrap:anywhere;background:#edf2f6;padding:16px;border-radius:6px;font-size:.85rem}}summary{{cursor:pointer;font-weight:600;padding:10px 0}}select{{font:inherit;padding:6px 12px;margin-left:10px}}.sub{{color:#516378}}.grid{{display:grid;grid-template-columns:repeat(3,1fr);gap:14px}}.metric{{background:#edf6f0;padding:13px;border-radius:7px}}.metric b{{display:block;font-size:1.6rem}}article{{scroll-margin-top:20px}}.tablewrap{{overflow-x:auto}}@media(max-width:650px){{.grid{{grid-template-columns:1fr}}main{{padding:15px 10px}}header,article,.panel{{padding:16px}}h1{{font-size:1.6rem}}}}@media print{{details{{display:block}}select{{display:none}}article{{break-inside:avoid}}body{{background:white}}}}
|
||||
</style><main>
|
||||
<style>.enlarge{{display:block;padding:0;border:0;background:none;cursor:zoom-in;width:100%;text-align:left}}dialog{{max-width:96vw;max-height:94vh;padding:14px;border:1px solid #98aabd;border-radius:9px}}dialog::backdrop{{background:#102333cc}}dialog img{{max-width:none;max-height:none}}dialog form{{position:sticky;top:0;text-align:right;margin-bottom:10px;background:white}}dialog button{{font:inherit;padding:5px 15px}}</style>
|
||||
<header><div class="eyebrow">ACPC 2026 · contest USB image</div><h1>IDE compiler verification</h1>
|
||||
<p>{e(data['verdict'])}</p><div class="grid"><div class="metric"><b>{passed}/{total}</b>language workflows passed</div><div class="metric"><b>7</b>IDEs on the real VM desktop</div><div class="metric"><b>3000000004</b>expected result for 3000000000 + 4</div></div>
|
||||
<p class="sub">{build}</p><p class="sub">Screenshots are embedded in this file. Click any image to inspect it, or download its PNG. No external service is needed to view this report.</p></header>
|
||||
<section class="panel"><h2>What was tested</h2><p>Each recorded workflow was launched through the IDE interface, built or interpreted the supplied source, accepted input typed on the VM keyboard and produced the expected result. The screenshots were opened and visually inspected. Configuration was installed by Ansible; compiler settings were not repaired manually during the final GUI pass.</p><p>C rejects accidental C++ compilation and requires C23. C++ requires C++23 and uses <code>std::println</code>. Java asserts the exact runtime version and the required VM flags. Python asserts PyPy and its version. Supplementary checks inspect static C/C++ executables and Java class versions.</p><ul>{notes}</ul></section>
|
||||
<section class="panel"><h2>Results</h2><div class="tablewrap"><table><thead><tr><th>IDE</th><th>Language</th><th>Result</th><th>Action tested</th></tr></thead><tbody>{''.join(rows)}</tbody></table></div></section>
|
||||
<p><label>Show screenshots for <select id="filter"><option value="all">All IDEs</option>{options}</select></label></p>
|
||||
{''.join(cards)}
|
||||
{instructions}
|
||||
<section class="panel"><h2>Supporting checks</h2>{''.join(logs)}{fixtures}</section>
|
||||
<section class="panel"><h2>Configuration references</h2><p>The installed IDE files and their serialized settings were also inspected. These primary references explain the supported configuration mechanisms.</p><ul>{sources}</ul><p class="sub">Report generated {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M UTC')}.</p></section>
|
||||
</main><dialog id="viewer"><form method="dialog"><button>Close</button></form><img alt=""></dialog><script>const filter=document.getElementById('filter');function applyFilter(){{document.querySelectorAll('article[data-ide]').forEach(a=>a.hidden=filter.value!=='all'&&a.dataset.ide!==filter.value)}}filter.addEventListener('change',applyFilter);window.addEventListener('pageshow',applyFilter);document.querySelectorAll('a[href^="#"]').forEach(a=>a.addEventListener('click',()=>{{document.getElementById('filter').value='all';document.querySelectorAll('article[data-ide]').forEach(a=>a.hidden=false)}}));const viewer=document.getElementById('viewer');document.querySelectorAll('.enlarge').forEach(button=>button.addEventListener('click',()=>{{const source=button.querySelector('img'),target=viewer.querySelector('img');target.src=source.src;target.alt=source.alt;viewer.showModal()}}));viewer.addEventListener('click',event=>{{if(event.target===viewer)viewer.close()}});</script></html>'''
|
||||
(report / 'index.html').write_text(page)
|
||||
print(f'{report / "index.html"}: {passed}/{total} passed; {len(page):,} bytes')
|
||||
Executable
+31
@@ -0,0 +1,31 @@
|
||||
#!/usr/bin/env python3
|
||||
# Builds test_report/index.html from the screenshots and test_report/captions.txt
|
||||
# (one line per screenshot: NAME | caption), plus test_report/feature_check.txt.
|
||||
import html, os
|
||||
os.chdir(os.path.join(os.path.dirname(os.path.abspath(__file__)), '..', 'test_report'))
|
||||
captions = {}
|
||||
if os.path.exists('captions.txt'):
|
||||
for line in open('captions.txt'):
|
||||
if '|' in line:
|
||||
name, text = line.split('|', 1)
|
||||
captions[name.strip()] = text.strip()
|
||||
names = sorted(f[:-4] for f in os.listdir('.') if f.endswith('.png'))
|
||||
order = [n for n in captions if n in names] + [n for n in names if n not in captions]
|
||||
out = ['<!DOCTYPE html><html><head><meta charset="utf-8"><title>ACPC 2026 image, test report</title>',
|
||||
'<style>body{font-family:sans-serif;max-width:1300px;margin:auto;padding:20px}'
|
||||
'img{max-width:100%;border:1px solid #999}figure{margin:0 0 40px 0}'
|
||||
'figcaption{font-size:1.1em;margin:6px 0}pre{background:#f4f4f4;padding:12px}</style></head><body>',
|
||||
'<h1>ACPC 2026 contest image: test report</h1>']
|
||||
if os.path.exists('intro.txt'):
|
||||
out.append('<p>%s</p>' % html.escape(open('intro.txt').read().strip()))
|
||||
if os.path.exists('technotes_check.txt'):
|
||||
out.append('<h2>Technical notes checklist</h2><pre>%s</pre>' % html.escape(open('technotes_check.txt').read()))
|
||||
if os.path.exists('feature_check.txt'):
|
||||
out.append('<h2>Feature checklist</h2><pre>%s</pre>' % html.escape(open('feature_check.txt').read()))
|
||||
out.append('<h2>Screens</h2>')
|
||||
for n in order:
|
||||
out.append('<figure><figcaption><b>%s</b> %s</figcaption><img src="%s.png"></figure>'
|
||||
% (html.escape(n), html.escape(captions.get(n, '')), n))
|
||||
out.append('</body></html>')
|
||||
open('index.html', 'w').write('\n'.join(out))
|
||||
print('test_report/index.html:', len(order), 'screens')
|
||||
Executable
+10
@@ -0,0 +1,10 @@
|
||||
#!/usr/bin/env python3
|
||||
# Sends one command to the QEMU monitor: monitor.py SOCKET 'command'
|
||||
# (screendump FILE, sendkey KEY, device_add ..., quit, ...)
|
||||
import socket, sys, time
|
||||
s = socket.socket(socket.AF_UNIX)
|
||||
s.connect(sys.argv[1])
|
||||
time.sleep(0.3)
|
||||
s.recv(65536)
|
||||
s.sendall((sys.argv[2] + "\n").encode())
|
||||
time.sleep(0.5)
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
#!/bin/bash
|
||||
# Screenshot of the test VM: tests/shot.sh NAME -> test_report/NAME.png
|
||||
cd "$(dirname "$0")/.."
|
||||
mkdir -p test_report tests/work
|
||||
python3 -I tests/monitor.py tests/work/monitor.sock "screendump $PWD/tests/work/shot.ppm"
|
||||
sleep 1
|
||||
python3 -I -c "from PIL import Image; Image.open('tests/work/shot.ppm').save('test_report/$1.png')"
|
||||
echo "test_report/$1.png"
|
||||
Executable
+82
@@ -0,0 +1,82 @@
|
||||
#!/bin/bash
|
||||
# Checks, from the team account in the test VM, the statements of the
|
||||
# technical notes for the teams (docs/TechNotes.tex) that can be checked on
|
||||
# the machine: tests/technotes_check.sh -> test_report/technotes_check.txt
|
||||
cd "$(dirname "$0")/.."
|
||||
mkdir -p test_report
|
||||
tests/vssh.sh team 'bash -s' <<'EOF' | tee test_report/technotes_check.txt
|
||||
set +e
|
||||
mkdir -p ~/tn && cd ~/tn
|
||||
pass=0; fail=0
|
||||
ok(){ echo "PASS $1"; pass=$((pass+1)); }; ko(){ echo "FAIL $1"; fail=$((fail+1)); }
|
||||
t(){ d=$1; shift; if eval "$@" >/dev/null 2>&1; then ok "$d"; else ko "$d"; fi; }
|
||||
echo "2 3" > input.txt
|
||||
cat > base.cpp <<'X'
|
||||
#include <cstdio>
|
||||
int main(){int a,b;scanf("%d %d",&a,&b);printf("%d\n",a+b);}
|
||||
X
|
||||
for ext in cpp cc C cxx c++; do cp base.cpp "p.$ext"; rm -f a.out
|
||||
t "C++ extension .$ext: compilecpp, then runcpp < input.txt" "compilecpp p.$ext && [ \"\$(runcpp < input.txt)\" = 5 ]"; done
|
||||
cat > p.c <<'X'
|
||||
#include <stdio.h>
|
||||
int main(void){int a,b;if(scanf("%d %d",&a,&b)!=2)return 1;printf("%d\n",a+b);return 0;}
|
||||
X
|
||||
rm -f a.out; t "C: compilec p.c, then runc < input.txt" "compilec p.c && [ \"\$(runc < input.txt)\" = 5 ]"
|
||||
rm -f a.out; t "compilegcc is another name for compilec" "compilegcc p.c && [ -x a.out ]"
|
||||
rm -f a.out; t "compileg++ is another name for compilecpp" "compileg++ p.cpp && [ -x a.out ]"
|
||||
t "programs are linked statically (-static)" "file a.out | grep -q 'statically linked'"
|
||||
t "compilec runs gcc-15 -g -O2 -o a.out -std=gnu23 -static ... -lm" "grep -q 'gcc-15 -g -O2 -o a.out -std=gnu23 -static' /usr/local/bin/compilec"
|
||||
t "compilecpp runs the Kattis command" "grep -q 'g++-15 -g -O2 -o a.out -std=gnu++23 -static -lrt -Wl,--whole-archive -lpthread -Wl,--no-whole-archive' /usr/local/bin/compilecpp"
|
||||
cat > Prog.java <<'X'
|
||||
import java.util.*;
|
||||
public class Prog{public static void main(String[] a){Scanner s=new Scanner(System.in);System.out.println(s.nextInt()+s.nextInt());}}
|
||||
X
|
||||
t "Java: compilejava Prog.java gives Prog.class, runjava Prog < input.txt" "compilejava Prog.java && [ -f Prog.class ] && [ \"\$(runjava Prog < input.txt)\" = 5 ]"
|
||||
cat > Mem.java <<'X'
|
||||
public class Mem{public static void main(String[] a){System.out.println(Runtime.getRuntime().maxMemory()/1048576);}}
|
||||
X
|
||||
compilejava Mem.java; m1=$(runjava Mem); m2=$(MEMLIM=1024 runjava Mem)
|
||||
t "runjava uses 2048 MB (max heap $m1 MB)" "[ $m1 -gt 1900 ] && [ $m1 -le 2048 ]"
|
||||
t "MEMLIM=1024 runjava uses 1024 MB (max heap $m2 MB)" "[ $m2 -gt 950 ] && [ $m2 -le 1024 ]"
|
||||
t "java and javac are 25.0.4.1" "java -version 2>&1 | grep -q '\"25.0.4.1\"' && javac -version 2>&1 | grep -q 25.0.4.1"
|
||||
printf 'a,b=map(int,input().split())\nprint(a+b)\n' > p.py; cp p.py p.py3
|
||||
t "Python: compilepython3 p.py, then runpython3 p.py < input.txt" "compilepython3 p.py && [ \"\$(runpython3 p.py < input.txt)\" = 5 ]"
|
||||
t "Python extension .py3 runs" "[ \"\$(runpython3 p.py3 < input.txt)\" = 5 ]"
|
||||
t "runpython3 runs PyPy 7.3.23 (Python 3.11.15)" "runpython3 -c 'import sys; assert sys.implementation.name==\"pypy\" and sys.pypy_version_info[:3]==(7,3,23) and sys.version_info[:3]==(3,11,15)'"
|
||||
printf 'print(1/0)\n' > rt.py; t "compilepython3 only checks the syntax (a run time error passes)" "compilepython3 rt.py"
|
||||
printf 'print(\n' > bad.py; t "compilepython3 reports a syntax error" "! compilepython3 bad.py"
|
||||
t "python3 is another interpreter, CPython 3.14" "python3 --version | grep -q 'Python 3.14'"
|
||||
t "gcc-15 and g++-15 are 15.2.0" "[ \$(gcc-15 -dumpfullversion) = 15.2.0 ] && [ \$(g++-15 -dumpfullversion) = 15.2.0 ]"
|
||||
cat > deep.cpp <<'X'
|
||||
#include <cstdio>
|
||||
int f(int n){return n?1+f(n-1):0;}
|
||||
int main(){printf("%d\n",f(10000000));}
|
||||
X
|
||||
t "runcpp: large stack, recursion depth 10 million" "compilecpp deep.cpp && [ \"\$(runcpp)\" = 10000000 ]"
|
||||
t "man 3 printf works" "man 3 printf | head -5 | grep -qi printf"
|
||||
t "pypy3 -m pydoc heapq works" "pypy3 -m pydoc heapq | head -3 | grep -q heapq"
|
||||
t "logged in as team, home /home/team" "[ \"\$(id -un)\" = team ] && [ \"\$HOME\" = /home/team ] && loginctl list-sessions --no-legend | grep -q 'team seat0'"
|
||||
for app in clion codeblocks Eclipse intellij-idea pycharm com.microsoft.VSCode gvim emacs geany org.kde.kate org.gnome.gedit; do
|
||||
t "in the Applications menu: $app" "[ -f /usr/share/applications/$app.desktop ]"; done
|
||||
t "vim and nano are installed" "command -v vim && command -v nano"
|
||||
t "DevDocs at https://devdocs/ answers" "curl -sk -o /dev/null https://devdocs/"
|
||||
for d in c cpp openjdk~25 python~3.11; do
|
||||
t "DevDocs serves the documentation of $d" "[ \$(curl -sk -o /dev/null -w %{http_code} https://devdocs/docs/$d/index.json) = 200 ]"; done
|
||||
t "DevDocs has no Kotlin documentation" "[ \$(curl -sk -o /dev/null -w %{http_code} https://devdocs/docs/kotlin~2.4/index.json) != 200 ]"
|
||||
t "offline contest help is readable from the documented menu" "test -r /usr/share/doc/acpc/help/index.html && grep -qx 'Exec=firefox file:///usr/share/doc/acpc/help/index.html' /usr/share/applications/acpc-help.desktop"
|
||||
t "offline help covers input, judging and every contest language" "python3 -c \"from pathlib import Path;s=Path('/usr/share/doc/acpc/help/index.html').read_text();assert all(x in s for x in ['Reading until EOF','Judgements','C and C++','Java main class','Python 3 and modules'])\""
|
||||
t "contest site reachable" "curl -s -o /dev/null -m 8 https://atlantic-canada26.kattis.com"
|
||||
t "Kattis status page reachable" "curl -s -o /dev/null -m 8 https://status.kattis.com"
|
||||
t "other websites blocked (en.cppreference.com)" "! curl -s -o /dev/null -m 8 https://en.cppreference.com"
|
||||
t "clock synchronized" "chronyc -n sources | grep -q '^\^\*'"
|
||||
t "time zone Atlantic (ADT/AST)" "date +%Z | grep -qE 'ADT|AST'"
|
||||
for source in p.c p.cpp Prog.java p.py; do
|
||||
t "IDE action acpc-solution builds and runs $source" "acpc-solution run $source < input.txt | tail -1 | grep -qx 5"
|
||||
done
|
||||
t "VS Code contest folder and task shortcut exist" "test -d ~/contest && grep -q f6 ~/.config/Code/User/keybindings.json"
|
||||
t "Eclipse named contest build/run action exists" "test -f ~/eclipse-workspace/.metadata/.plugins/org.eclipse.debug.core/.launches/ACPC\ Build\ and\ run.launch"
|
||||
t "Geany Java uses runjava for the documented VM options" "grep -q '^EX_00_CM=runjava' /usr/share/geany/filedefs/filetypes.java"
|
||||
t "CLion includes Python with the PyPy SDK" "grep -q /usr/local/bin/pypy3 ~/.config/JetBrains/CLion*/options/jdk.table.xml"
|
||||
cd ~ && rm -rf ~/tn
|
||||
echo; echo "$pass passed, $fail failed"
|
||||
EOF
|
||||
@@ -0,0 +1,3 @@
|
||||
[Service]
|
||||
# Test VM only (tests/check_vm.sh). Accepts SSH so the test tools can log in.
|
||||
ExecStartPost=/usr/sbin/nft insert rule inet filter input tcp dport 22 accept
|
||||
Executable
+36
@@ -0,0 +1,36 @@
|
||||
#!/usr/bin/env python3
|
||||
# Types text into the VM through the QEMU monitor (US keyboard).
|
||||
# Usage: type.py MONITOR_SOCKET 'text' (newline = Enter; keys: {{ctrl-s}}, {{esc}}, ...)
|
||||
import re, socket, sys, time
|
||||
|
||||
SHIFTED = {'!': '1', '@': '2', '#': '3', '$': '4', '%': '5', '^': '6', '&': '7', '*': '8',
|
||||
'(': '9', ')': '0', '_': 'minus', '+': 'equal', '{': 'bracket_left',
|
||||
'}': 'bracket_right', '|': 'backslash', ':': 'semicolon', '"': 'apostrophe',
|
||||
'<': 'comma', '>': 'dot', '?': 'slash', '~': 'grave_accent'}
|
||||
PLAIN = {' ': 'spc', '\n': 'ret', '\t': 'tab', '-': 'minus', '=': 'equal', '[': 'bracket_left',
|
||||
']': 'bracket_right', '\\': 'backslash', ';': 'semicolon', "'": 'apostrophe',
|
||||
',': 'comma', '.': 'dot', '/': 'slash', '`': 'grave_accent'}
|
||||
|
||||
def keys(text):
|
||||
for token in re.split(r'(\{\{[a-z0-9_-]+\}\})', text):
|
||||
if token.startswith('{{') and token.endswith('}}'):
|
||||
yield token[2:-2]
|
||||
continue
|
||||
for c in token:
|
||||
if c in PLAIN:
|
||||
yield PLAIN[c]
|
||||
elif c in SHIFTED:
|
||||
yield 'shift-' + SHIFTED[c]
|
||||
elif c.isupper():
|
||||
yield 'shift-' + c.lower()
|
||||
elif c.isalnum():
|
||||
yield c
|
||||
|
||||
s = socket.socket(socket.AF_UNIX)
|
||||
s.connect(sys.argv[1])
|
||||
time.sleep(0.2)
|
||||
text = open(sys.argv[2][1:]).read() if sys.argv[2].startswith('@') else sys.argv[2]
|
||||
for k in keys(text):
|
||||
s.sendall(('sendkey %s\n' % k).encode())
|
||||
time.sleep(0.06)
|
||||
time.sleep(0.3)
|
||||
Executable
+19
@@ -0,0 +1,19 @@
|
||||
#!/bin/bash
|
||||
# Plugs a USB stick (32 MB, label TEAMUSB, one file) into the test VM started
|
||||
# with --usb: tests/usb.sh plug|unplug
|
||||
cd "$(dirname "$0")/.."
|
||||
W=tests/work
|
||||
case "$1" in
|
||||
plug)
|
||||
mkdir -p "$W/usb/content"
|
||||
echo "prepared code library" > "$W/usb/content/lib.cpp"
|
||||
truncate -s 32M "$W/usb/usb.img"
|
||||
mkfs.ext4 -q -F -L TEAMUSB -d "$W/usb/content" "$W/usb/usb.img"
|
||||
n=$(date +%s)
|
||||
python3 -I tests/monitor.py "$W/monitor.sock" "drive_add 0 if=none,id=ud$n,file=$PWD/$W/usb/usb.img,format=raw"
|
||||
python3 -I tests/monitor.py "$W/monitor.sock" "device_add usb-storage,bus=xhci.0,drive=ud$n,id=usbstick"
|
||||
;;
|
||||
unplug)
|
||||
python3 -I tests/monitor.py "$W/monitor.sock" "device_del usbstick"
|
||||
;;
|
||||
esac
|
||||
Executable
+11
@@ -0,0 +1,11 @@
|
||||
#!/bin/bash
|
||||
# Runs a command in the test VM: tests/vssh.sh team|icpc 'command'
|
||||
# Passwords: TEAM_PASS (default contest) and ADMIN_PASS (password given to build.sh).
|
||||
user=$1; shift
|
||||
case "$user" in
|
||||
team) pass=${TEAM_PASS:-contest} ;;
|
||||
icpc) pass=${ADMIN_PASS:?set ADMIN_PASS to the administrator password} ;;
|
||||
*) echo "user must be team or icpc" >&2; exit 2 ;;
|
||||
esac
|
||||
exec sshpass -p "$pass" ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
|
||||
-o LogLevel=ERROR -o ConnectTimeout=5 -p 2222 "$user@127.0.0.1" "$@"
|
||||
Reference in new issue
Block a user