Bug 76921

Summary: raw1394_arm_register() fails on kernel 3.14.x
Product: Drivers Reporter: Fabien Spindler (Fabien.Spindler)
Component: IEEE1394Assignee: drivers_ieee1394
Status: CLOSED CODE_FIX    
Severity: normal CC: stefanr
Priority: P1    
Hardware: x86-64   
OS: Linux   
Kernel Version: 3.14 Subsystem:
Regression: Yes Bisected commit-id:
Attachments: test_arm_register.c

Description Fabien Spindler 2014-05-26 13:15:48 UTC
Created attachment 137411 [details]
test_arm_register.c

Dear,

I have a device connected throw firewire to a computer running Linux (Fedora 20 or Ubuntu 14.04). I use raw1394_arm_register() to map an area starting at 0x100000000000LL.

When the kernel version is 3.13.x (tested with 3.13.0-24 ubuntu x86_64, 3.13.6-200-fc20.x86_64, 3.13.10-200-fc20.x86_64) raw1394_arm_register() is able to register.

But with a kernel 3.14.x (tested on 3.14.3-200.fc20.x86_64, 3.14.4-200.fc20.x86_64) raw1394_arm_register() returns Device or resource busy.

Below I have attached a sample test.

On 3.14.x it produces the following output:

$ ./test_arm_register 
Start address: 0x1000000000
Number of board detected: 1
  nodes on bus:  2, card name: /dev/fw1
set port: 0
error allocation arm register: Device or resource busy
error num 0, 57005

I hope that a solution could be found to fix that issue.

Fabien

--- begin test_arm_register.c ---
#include <stdio.h>
#include <libraw1394/raw1394.h>

#define ADDR_HI	((nodeaddr_t)0x10ULL)
#define ADDR_LO	((nodeaddr_t)0x0ULL)

int main()
{
  raw1394handle_t handle = NULL;
  int number_of_1394cards;
  int stt;
  int i;

  nodeaddr_t addr = (ADDR_HI<<32) + ADDR_LO; 
  size_t length = 0x1000; 
  octlet_t pointer_to_handle = NULL;
  struct raw1394_portinfo ports_information[16];
  int port = 0;
  printf("Start address: 0x%lX\n", addr);

  handle = raw1394_new_handle();
  if (! handle) {
    perror("coundn't get handle");
    return -1;
  }

  number_of_1394cards = raw1394_get_port_info(handle, ports_information, 16);
  if (number_of_1394cards < 0) { 
    perror("no card detected");
    return -1;
  }

  printf("Number of board detected: %d\n", number_of_1394cards);  
  // Print all cards informations
  for (i = 0; i < number_of_1394cards; i++) {
    printf("  nodes on bus: %2d, card name: %s\n",
	   ports_information[i].nodes,
	   ports_information[i].name
	   );
  }

  printf("set port: %d\n", port);
  if (raw1394_set_port(handle, port) < 0) {
    perror("couldn't set port");
    return -1;
  }

  stt = raw1394_arm_register(handle,
			     addr,
			     length,
			     NULL,
			     pointer_to_handle,
			     RAW1394_ARM_READ | RAW1394_ARM_WRITE,
			     RAW1394_ARM_READ | RAW1394_ARM_WRITE,
			     0
			     );

  if (stt < 0) {
    perror("error allocation arm register");
    raw1394_errcode_t error = raw1394_get_errcode (handle);
    printf("error num %d, %d\n", error, raw1394_errcode_to_errno(error) );
    return -1;
  }

  printf("arm register successfully allocated\n");

  return 0;

}
--- end test_arm_register.c ---


--- begin Makefile ---
CC=gcc
CFLAGS=-g
LIBS=-lraw1394

all: test_arm_register

test_arm_register: test_arm_register.c
	$(CC) $(CFLAGS) $< -o $@ $(LIBS)

clean:
	rm -f test_arm_register
	rm *~
--- end Makefile ---
Comment 1 Stefan Richter 2014-05-26 13:59:28 UTC
Thanks for the report.

Since kernel v3.14, the lowermost 128 TB of the 48bit address space (i.e. 0x0000'0000'0000...0x7fff'ffff'ffff inclusive) are set aside for physical DMA.  http://git.kernel.org/linus/fcd46b34425d

On a related note, IEEE 1212 specifies the lowermost 256 TB minus 512 MB (i.e the range at 0x0000'0000'0000...0xffff'dfff'ffff inclusive) as "memory space" ("read and write transactions do not have side effects"), the next 256 MB (i.e. 0xffff'e000'0000...0xffff'efff'ffff inclusive) as "private space", "allocated for vendor-dependent uses *local* to the node" (emphasis mine), and the remaining 256 MB (i.e. 0xffff'f000'0000...0xffff'ffff'ffff inclusive) as "register space".  Within the latter, the first 2048 bytes are reserved for IEEE 1212/ IEEE 1394 general registers, meaning that the range above this (called "units space", 0xffff'f000'0800...0xffff'ffff'ffff inclusive) is available for unit architecture defined registers.

Do you require to put your registers already close above the 4 GB barrier, e.g. because you need to interoperate with an unmodifiable system at this range?  If so, then we need to make the new v3.14 behavior optional instead of mandatory default.  Or can you work with respectively higher addresses easily?
Comment 2 Fabien Spindler 2014-05-28 12:45:34 UTC
Stefan thanks for you feedback.

I tried to modify test_arm_register.c by using the private space and it works well on both 3.13 and 3.14 kernels.

But since I don't have the source code of my device (it is a robot) I'm not able to change the address for the moment. I have contacted the vendor to see if they can change that.

Fabien
Comment 3 Stefan Richter 2014-05-28 14:48:58 UTC
When we added the 128 TB physical DMA support by commit fcd46b34425d, we were already aware of one vendor using IEEE 1212 Memory Space for registers:  Echo Fireworks protocol for some audio devices, with registers at [0xecc0'0000'0000...0xecc0'0000'0200) at the device and [0xecc0'8000'0000...0xecc0'8000'0200) at the controlling PC.  Therefore we kept a little bit of a distance from these addresses which are still well within the OHCI-1394 maximum allowed physical range.

Learning that the [0x0010'0000'0000...0x0010'0001'0000) range which you are using was defined by a device vendor, rather than being random Joe Programmer's ad-hoc choice of address, convinces me that we need to revert to the 3.13 and earlier status:
  - Request reception above 0x0001'0000'0000 supported,
  - physical DMA above 0x0001'0000'0000 unsupported.
Going forward, we can implement _optional_ support for physical DMA above 0x0001'0000'0000 (again preventing request reception at those addresses), subject to the local administrator's active choice.  Obviously, this will make the physical DMA feature less powerful, because it will be unavailable despite kernel support if the target system isn't explicitly configured for it.  However, request reception at those lower ranges was a working feature before, there the new physical DMA feature needs to take a back seat.

Maybe your device vendor will agree to change the addresses for you, but others may not be as lucky.
Comment 4 Fabien Spindler 2014-05-30 07:10:14 UTC
With the vendor we were able to do the changes to make the device working with addresses in the private space (0xffff'e000'0000...0xffff'efff'ffff inclusive).

I agree with you that it seems better for compat to extend the DMA optionally.

For me the ticket can be closed.

Thanks
Comment 5 Stefan Richter 2014-05-30 19:59:55 UTC
Thanks for looking into a solution from your end together with the vendor.
Meanwhile, a fix is pending for release in 3.15 (-rc8 or -final) and for backmerge into the 3.14.x stable series:
http://git.kernel.org/linus/2fe2023adf69