102 lines
3.6 KiB
ReStructuredText
102 lines
3.6 KiB
ReStructuredText
|
=====================
|
||
|
|
DAWR issues on POWER9
|
||
|
|
=====================
|
||
|
|
|
||
|
|
On older POWER9 processors, the Data Address Watchpoint Register (DAWR) can
|
||
|
|
cause a checkstop if it points to cache inhibited (CI) memory. Currently Linux
|
||
|
|
has no way to distinguish CI memory when configuring the DAWR, so on affected
|
||
|
|
systems, the DAWR is disabled.
|
||
|
|
|
||
|
|
Affected processor revisions
|
||
|
|
============================
|
||
|
|
|
||
|
|
This issue is only present on processors prior to v2.3. The revision can be
|
||
|
|
found in /proc/cpuinfo::
|
||
|
|
|
||
|
|
processor : 0
|
||
|
|
cpu : POWER9, altivec supported
|
||
|
|
clock : 3800.000000MHz
|
||
|
|
revision : 2.3 (pvr 004e 1203)
|
||
|
|
|
||
|
|
On a system with the issue, the DAWR is disabled as detailed below.
|
||
|
|
|
||
|
|
Technical Details:
|
||
|
|
==================
|
||
|
|
|
||
|
|
DAWR has 6 different ways of being set.
|
||
|
|
1) ptrace
|
||
|
|
2) h_set_mode(DAWR)
|
||
|
|
3) h_set_dabr()
|
||
|
|
4) kvmppc_set_one_reg()
|
||
|
|
5) xmon
|
||
|
|
|
||
|
|
For ptrace, we now advertise zero breakpoints on POWER9 via the
|
||
|
|
PPC_PTRACE_GETHWDBGINFO call. This results in GDB falling back to
|
||
|
|
software emulation of the watchpoint (which is slow).
|
||
|
|
|
||
|
|
h_set_mode(DAWR) and h_set_dabr() will now return an error to the
|
||
|
|
guest on a POWER9 host. Current Linux guests ignore this error, so
|
||
|
|
they will silently not get the DAWR.
|
||
|
|
|
||
|
|
kvmppc_set_one_reg() will store the value in the vcpu but won't
|
||
|
|
actually set it on POWER9 hardware. This is done so we don't break
|
||
|
|
migration from POWER8 to POWER9, at the cost of silently losing the
|
||
|
|
DAWR on the migration.
|
||
|
|
|
||
|
|
For xmon, the 'bd' command will return an error on P9.
|
||
|
|
|
||
|
|
Consequences for users
|
||
|
|
======================
|
||
|
|
|
||
|
|
For GDB watchpoints (ie 'watch' command) on POWER9 bare metal , GDB
|
||
|
|
will accept the command. Unfortunately since there is no hardware
|
||
|
|
support for the watchpoint, GDB will software emulate the watchpoint
|
||
|
|
making it run very slowly.
|
||
|
|
|
||
|
|
The same will also be true for any guests started on a POWER9
|
||
|
|
host. The watchpoint will fail and GDB will fall back to software
|
||
|
|
emulation.
|
||
|
|
|
||
|
|
If a guest is started on a POWER8 host, GDB will accept the watchpoint
|
||
|
|
and configure the hardware to use the DAWR. This will run at full
|
||
|
|
speed since it can use the hardware emulation. Unfortunately if this
|
||
|
|
guest is migrated to a POWER9 host, the watchpoint will be lost on the
|
||
|
|
POWER9. Loads and stores to the watchpoint locations will not be
|
||
|
|
trapped in GDB. The watchpoint is remembered, so if the guest is
|
||
|
|
migrated back to the POWER8 host, it will start working again.
|
||
|
|
|
||
|
|
Force enabling the DAWR
|
||
|
|
=======================
|
||
|
|
Kernels (since ~v5.2) have an option to force enable the DAWR via::
|
||
|
|
|
||
|
|
echo Y > /sys/kernel/debug/powerpc/dawr_enable_dangerous
|
||
|
|
|
||
|
|
This enables the DAWR even on POWER9.
|
||
|
|
|
||
|
|
This is a dangerous setting, USE AT YOUR OWN RISK.
|
||
|
|
|
||
|
|
Some users may not care about a bad user crashing their box
|
||
|
|
(ie. single user/desktop systems) and really want the DAWR. This
|
||
|
|
allows them to force enable DAWR.
|
||
|
|
|
||
|
|
This flag can also be used to disable DAWR access. Once this is
|
||
|
|
cleared, all DAWR access should be cleared immediately and your
|
||
|
|
machine once again safe from crashing.
|
||
|
|
|
||
|
|
Userspace may get confused by toggling this. If DAWR is force
|
||
|
|
enabled/disabled between getting the number of breakpoints (via
|
||
|
|
PTRACE_GETHWDBGINFO) and setting the breakpoint, userspace will get an
|
||
|
|
inconsistent view of what's available. Similarly for guests.
|
||
|
|
|
||
|
|
For the DAWR to be enabled in a KVM guest, the DAWR needs to be force
|
||
|
|
enabled in the host AND the guest. For this reason, this won't work on
|
||
|
|
POWERVM as it doesn't allow the HCALL to work. Writes of 'Y' to the
|
||
|
|
dawr_enable_dangerous file will fail if the hypervisor doesn't support
|
||
|
|
writing the DAWR.
|
||
|
|
|
||
|
|
To double check the DAWR is working, run this kernel selftest:
|
||
|
|
|
||
|
|
tools/testing/selftests/powerpc/ptrace/ptrace-hwbreak.c
|
||
|
|
|
||
|
|
Any errors/failures/skips mean something is wrong.
|