Bug 205459

Summary: mips: bpf: test_bpf failures, eBPF JIT on mips32 outputs invalid 64-bit insns
Product: Networking Reporter: Tony Ambardar (itugrok)
Component: OtherAssignee: Stephen Hemminger (stephen)
Status: RESOLVED CODE_FIX    
Severity: high CC: daniel
Priority: P1    
Hardware: Mips32   
OS: Linux   
Kernel Version: 5.2.17 Subsystem:
Regression: No Bisected commit-id:
Attachments: EXCEPTION/failures: kernel 5.2.17/mips32 (Debian 10.1)

Description Tony Ambardar 2019-11-07 06:41:21 UTC
Created attachment 285809 [details]
EXCEPTION/failures: kernel 5.2.17/mips32 (Debian 10.1)

Summary:
========

Linux 5.2.x added an eBPF JIT for MIPS32 (yay!). Based on discussion of the original submission (https://www.spinics.net/lists/mips/msg77008.html) I expected that:

  (1) all tests from module test_bpf.ko would pass, and
  (2) any previously JITed tests (i.e. cBPF) would still be JITed.

However, I can't reproduce the above based on my testing as per the attached log.

Point (2) doesn't stand since the first ~30 tests are not JITed, but were previously cBPF JITed for the most part.

As for point (1), the full test set doesn't complete, but errors out early on with a "Reserved instruction in kernel code[#1]" error. Manually hopping through some of the tests yields the same error for many:

  #68 ALU_MOV_K: 0x0000ffffffff0000 = 0x00000000ffffffff jited:1
  #73 ALU_ADD_X: 1 + 2 = 3 jited:1
  #74 ALU_ADD_X: 1 + 4294967294 = 4294967295 jited:1
  #75 ALU_ADD_X: 2 + 4294967294 = 0 jited:1
  #79 ALU_ADD_K: 1 + 2 = 3 jited:1
  (.. and so on ...)

Disassembling the JITed code for test #68 shows incorrect MIPS64 instructions:

  24 03 00 20     li	v1,32
  34 05 ff ff     li	a1,0xffff
  00 05 2c 38     dsll	a1,a1,0x10      <=== MIPS64 insn
  34 a5 ff ff     ori	a1,a1,0xffff
  00 05 2c 38     dsll	a1,a1,0x10      <=== MIPS64 insn
  34 06 ff ff     li	a2,0xffff
  00 06 34 38     dsll	a2,a2,0x10      <=== MIPS64 insn
  34 c6 ff ff     ori	a2,a2,0xffff

Since this was tested in the past, I'm really hoping there's a simple solution to these problems, or else a case of "operator error". A review by someone more knowledgeable with the MIPS32 eBPF JIT would be appreciated. 


Steps to Reproduce:
===================

  # sysctl net.core.bpf_jit_enable=1
  # modprobe test_bpf
  <Kernel log with "Reserved instruction" exception>


Affected Systems Tested:
========================

  Debian 10.1 on QEMU/malta(mips32_be) [distro kernel 5.2.17-1~bpo10+1 (2019-09-30)]


Kernel Logs:
============

Boot log with test results up to first failure is attached.
Comment 1 Daniel Borkmann 2019-11-07 07:31:08 UTC
Thanks for your report! Could you send this report to the kernel mailing list with mips folks in Cc (report should make the the following are in Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, linux-mips@vger.kernel.org, paulburton@kernel.org, jhogan@kernel.org)?
Comment 2 Tony Ambardar 2019-11-07 09:09:12 UTC
Ah, sorry, I didn't CC myself so don't have an email I can easily forward. Would you mind forwarding the first email? If not, I'll try to format another mail.

And in the future, should I email the maintainers/lists directly rather than using Bugzilla? I had tried to add "paulburton@kernel.org" to Bugzilla's "CC" field but it wouldn't accept it...
Comment 3 Tony Ambardar 2020-05-06 02:45:03 UTC
Upstream workaround was to disable the MIPS32 eBPF JIT and restore the cBPF JIT to avoid any cBPF performance regression. Development of a working MIPS32 eBPF JIT is still pending.