Created attachment 202201 [details]
Kernel config

Created attachment 202211 [details]
config-4.4.0

HI Mike,

Thank you for looking into this issue.
I reproduced this issue  this using both mainline compiled kernel and the one posted by El Repo.  I used both of these kernels with RHEL 7.2, both kernels showed exactly the same issue. When I rollback to the original kernel 3.10.0-327.4.4, I do not see the issue.

As per your request attaching kernel config file. Please let me know if you need any additional info or if you want to take a look at the router, I can setup a webex meeting to show you the issue.

Thank you

Serguei




Serguei Bezverkhi,
TECHNICAL LEADER.SERVICES
Global SP Services
sbezverk@cisco.com
Phone: +1 416 306 7312
Mobile: +1 514 234 7374

CCIE (R&S,SP,Sec) - #9527

Cisco.com



 Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here for Company Registration Information.




-----Original Message-----
From: linux-scsi-owner@vger.kernel.org [mailto:linux-scsi-owner@vger.kernel.org] On Behalf Of Mike Christie
Sent: Thursday, January 28, 2016 8:53 PM
To: bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org
Subject: Re: [Bug 111441] New: iscsi fails to attach to targets

On 01/28/2016 04:51 PM, bugzilla-daemon@bugzilla.kernel.org wrote:
> https://bugzilla.kernel.org/show_bug.cgi?id=111441
> 
>             Bug ID: 111441
>            Summary: iscsi fails to attach to targets
>            Product: IO/Storage
>            Version: 2.5
>     Kernel Version: 4.4.0-1
>           Hardware: x86-64
>                 OS: Linux
>               Tree: Mainline
>             Status: NEW


> 4.4.0-1.el7.elrepo.x86_64 #1

I have not seen this oops before. We saw similar ones around 5 or 6 years ago, but they were due to some sysfs or block or scsi issue (I cannot remember exacty) and fixed there.

Is this a distro kernel or mainline. BZ says mainline, but the kernel name in bug looks like a red hat related one. Where did you get it?

I do not hit this in 4.4 mainline. Send me your kernel .config, so I can make sure I have the same options.

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

Additional tracebacks:

[   25.799382] WARNING: CPU: 18 PID: 1482 at block/genhd.c:626 add_disk+0x443/0x4a0()
[   25.799383] Modules linked in: iscsi_tcp libiscsi_tcp 8021q garp mrp rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod openvswitch bonding ib_srp scsi_transport_srp ib_ipoib nf_defrag_ipv6 rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_sa ib_mad usnic_verbs ib_core xt_comment xt_multiport iptable_filter ib_addr iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 x86_pkg_temp_thermal nf_nat_ipv4 intel_powerclamp nf_nat coretemp kvm_intel nf_conntrack kvm iptable_mangle irqbypass crct10dif_pclmul crc32_pclmul nfsd aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ses enclosure sb_edac sg auth_rpcgss edac_core iTCO_wdt ipmi_devintf iTCO_vendor_support nfs_acl lockd shpchp pcspkr lpc_ich ipmi_si ipmi_msghandler mfd_core wmi acpi_power_meter grace
[   25.799435]  sunrpc br_netfilter bridge stp llc dm_multipath ip_tables xfs libcrc32c sd_mod mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ixgbe(O) drm fnic i40e libfcoe igb megaraid_sas enic libfc crc32c_intel vxlan ip6_udp_tunnel udp_tunnel ptp scsi_transport_fc i2c_algo_bit dca i2c_core pps_core dm_mirror dm_region_hash dm_log dm_mod
[   25.799462] CPU: 18 PID: 1482 Comm: kworker/u82:3 Tainted: G           O    4.4.0 #1
[   25.799464] Hardware name: Cisco Systems Inc UCSC-C240-M3S/UCSC-C240-M3S, BIOS C240M3.2.0.8.0.071620152208 07/16/2015
[   25.799470] Workqueue: events_unbound async_run_entry_fn
[   25.799473]  0000000000000000 00000000866f6387 ffff885f9d34bcf8 ffffffff81329090
[   25.799476]  0000000000000000 ffff885f9d34bd30 ffffffff810862c6 ffff885e6c3a8800
[   25.799478]  ffff885e6c3a8880 ffff885f9c59ea80 ffff885e6c3a880c ffff885e6bc78000
[   25.799481] Call Trace:
[   25.799492]  [<ffffffff81329090>] dump_stack+0x44/0x64
[   25.799496]  [<ffffffff810862c6>] warn_slowpath_common+0x86/0xc0
[   25.799499]  [<ffffffff8108640a>] warn_slowpath_null+0x1a/0x20
[   25.799502]  [<ffffffff8130ef23>] add_disk+0x443/0x4a0
[   25.799508]  [<ffffffffa0086a35>] sd_probe_async+0x115/0x1d0 [sd_mod]
[   25.799511]  [<ffffffff810a70fa>] async_run_entry_fn+0x4a/0x140
[   25.799515]  [<ffffffff8109e09c>] process_one_work+0x14c/0x3c0
[   25.799517]  [<ffffffff8109e8d4>] worker_thread+0x114/0x470
[   25.799520]  [<ffffffff8109e7c0>] ? rescuer_thread+0x310/0x310
[   25.799524]  [<ffffffff810a43d8>] kthread+0xd8/0xf0
[   25.799527]  [<ffffffff810a4300>] ? kthread_park+0x60/0x60
[   25.799532]  [<ffffffff8169490f>] ret_from_fork+0x3f/0x70
[   25.799535]  [<ffffffff810a4300>] ? kthread_park+0x60/0x60
[   25.799538] ---[ end trace 5d59079faf7d2510 ]---
[   25.799568] sd 3:0:0:2: [sdc] Attached SCSI disk
[   25.808827] scsi 3:0:0:0: Unexpected response from lun 2 while scanning, scan aborted
[   25.810073] scsi host6: iSCSI Initiator over TCP/IP
[   25.811250] scsi 4:0:0:3: Direct-Access     LIO-ORG  san-disk-3       4.0  PQ: 0 ANSI: 5
[   25.811776] sd 4:0:0:3: alua: supports implicit and explicit TPGS
[   25.811914] sd 4:0:0:3: [sdd] 9765625856 512-byte logical blocks: (5.00 TB/4.54 TiB)
[   25.811958] sd 4:0:0:3: alua: No target port descriptors found
[   25.811961] sd 4:0:0:3: alua: Attach failed (-22)
[   25.811964] sd 4:0:0:3: failed to add device handler: -22
[   25.812001] sd 4:0:0:3: [sdd] Write Protect is off
[   25.812005] sd 4:0:0:3: [sdd] Mode Sense: 00 00 00 00
[   25.812009] sd 4:0:0:3: [sdd] Asking for cache data failed
[   25.812011] sd 4:0:0:3: [sdd] Assuming drive cache: write through
[   25.815398] sd 4:0:0:3: [sdd] Attached SCSI disk
[   25.819131] ixgbe 0000:04:00.1: SR-IOV enabled with 8 VFs
[   25.819134] ixgbe 0000:04:00.1: configure port vlans to keep your VFs secure
[   25.824709] scsi 4:0:0:0: Unexpected response from lun 3 while scanning, scan aborted
[   25.832694] scsi 6:0:0:4: Direct-Access     LIO-ORG  san-disk-4       4.0  PQ: 0 ANSI: 5
[   25.832921] scsi 5:0:0:1: Direct-Access     LIO-ORG  san-disk-1       4.0  PQ: 0 ANSI: 5
[   25.833195] sd 6:0:0:4: alua: supports implicit and explicit TPGS
[   25.833360] sd 6:0:0:4: alua: No target port descriptors found
[   25.833363] sd 6:0:0:4: alua: Attach failed (-22)
[   25.833365] sd 6:0:0:4: failed to add device handler: -22
[   25.833394] sd 5:0:0:1: alua: supports implicit and explicit TPGS
[   25.851814] sd 6:0:0:4: [sde] 10742171648 512-byte logical blocks: (5.49 TB/5.00 TiB)
[   25.851820] sd 5:0:0:1: [sdf] 7812499389 512-byte logical blocks: (3.99 TB/3.63 TiB)
[   25.851821] sd 6:0:0:4: [sde] Write Protect is off
[   25.851823] sd 6:0:0:4: [sde] Mode Sense: 00 00 00 00
[   25.851825] sd 6:0:0:4: [sde] Asking for cache data failed
[   25.851827] sd 6:0:0:4: [sde] Assuming drive cache: write through
[   25.851843] sd 5:0:0:1: alua: No target port descriptors found
[   25.851847] sd 5:0:0:1: alua: Attach failed (-22)
[   25.851849] sd 5:0:0:1: failed to add device handler: -22
[   25.851946] sd 5:0:0:1: [sdf] Write Protect is off
[   25.851948] sd 5:0:0:1: [sdf] Mode Sense: 00 00 00 00
[   25.851949] sd 5:0:0:1: [sdf] Asking for cache data failed
[   25.851951] sd 5:0:0:1: [sdf] Assuming drive cache: write through
[   25.852160] sd 6:0:0:4: [sde] Attached SCSI disk
[   25.852172] sd 5:0:0:1: [sdf] Attached SCSI disk
[   25.854892] scsi 6:0:0:0: Unexpected response from lun 4 while scanning, scan aborted
[   25.862806] scsi 5:0:0:0: Unexpected response from lun 1 while scanning, scan aborted
[   25.890721] ixgbe 0000:04:00.0 enp4s0f0: detected SFP+: 7
[   25.919701] pci 0000:04:10.1: [8086:10ed] type 00 class 0x020000
[   25.919738] pci 0000:04:10.1: can't set Max Payload Size to 256; if necessary, use "pci=pcie_bus_safe" and report a bug
[   25.919863] iommu: Adding device 0000:04:10.1 to group 88
[   25.919920] pci 0000:04:10.3: [8086:10ed] type 00 class 0x020000
[   25.919951] pci 0000:04:10.3: can't set Max Payload Size to 256; if necessary, use "pci=pcie_bus_safe" and report a bug
[   25.920013] iommu: Adding device 0000:04:10.3 to group 89
[   25.920039] pci 0000:04:10.5: [8086:10ed] type 00 class 0x020000
[   25.920069] pci 0000:04:10.5: can't set Max Payload Size to 256; if necessary, use "pci=pcie_bus_safe" and report a bug
[   25.920127] iommu: Adding device 0000:04:10.5 to group 90
[   25.920151] pci 0000:04:10.7: [8086:10ed] type 00 class 0x020000
[   25.920182] pci 0000:04:10.7: can't set Max Payload Size to 256; if necessary, use "pci=pcie_bus_safe" and report a bug
[   25.920239] iommu: Adding device 0000:04:10.7 to group 91
[   25.920263] pci 0000:04:11.1: [8086:10ed] type 00 class 0x020000
[   25.920293] pci 0000:04:11.1: can't set Max Payload Size to 256; if necessary, use "pci=pcie_bus_safe" and report a bug
[   25.920349] iommu: Adding device 0000:04:11.1 to group 92
[   25.920373] pci 0000:04:11.3: [8086:10ed] type 00 class 0x020000
[   25.920403] pci 0000:04:11.3: can't set Max Payload Size to 256; if necessary, use "pci=pcie_bus_safe" and report a bug
[   25.920459] iommu: Adding device 0000:04:11.3 to group 93
[   25.920485] pci 0000:04:11.5: [8086:10ed] type 00 class 0x020000
[   25.920515] pci 0000:04:11.5: can't set Max Payload Size to 256; if necessary, use "pci=pcie_bus_safe" and report a bug
[   25.920570] iommu: Adding device 0000:04:11.5 to group 94
[   25.920594] pci 0000:04:11.7: [8086:10ed] type 00 class 0x020000
[   25.920624] pci 0000:04:11.7: can't set Max Payload Size to 256; if necessary, use "pci=pcie_bus_safe" and report a bug
[   25.920698] iommu: Adding device 0000:04:11.7 to group 95
[   25.921365] ixgbe 0000:04:00.1: removed PHC on enp4s0f1
[   26.020448] ixgbe 0000:04:00.1: registered PHC device on enp4s0f1
[   26.185674] ixgbe 0000:04:00.1 enp4s0f1: detected SFP+: 4
[   26.380954] i40e 0000:06:00.0 enp6s0f0: adding 68:05:ca:37:26:90 vid=0
[   26.392266] i40e 0000:06:00.0: Allocating 8 VFs.
[   26.492733] pci 0000:06:02.0: [8086:154c] type 00 class 0x020000
[   26.492785] pci 0000:06:02.0: Max Payload Size set to 256 (was 128, max 2048)
[   26.493016] iommu: Adding device 0000:06:02.0 to group 96
[   26.493114] pci 0000:06:02.1: [8086:154c] type 00 class 0x020000
[   26.493159] pci 0000:06:02.1: Max Payload Size set to 256 (was 128, max 2048)
[   26.493259] iommu: Adding device 0000:06:02.1 to group 97
[   26.493309] pci 0000:06:02.2: [8086:154c] type 00 class 0x020000
[   26.493352] pci 0000:06:02.2: Max Payload Size set to 256 (was 128, max 2048)
[   26.493452] iommu: Adding device 0000:06:02.2 to group 98
[   26.493501] pci 0000:06:02.3: [8086:154c] type 00 class 0x020000
[   26.493545] pci 0000:06:02.3: Max Payload Size set to 256 (was 128, max 2048)
[   26.493655] iommu: Adding device 0000:06:02.3 to group 99
[   26.493707] pci 0000:06:02.4: [8086:154c] type 00 class 0x020000
[   26.493751] pci 0000:06:02.4: Max Payload Size set to 256 (was 128, max 2048)
[   26.493856] iommu: Adding device 0000:06:02.4 to group 100
[   26.493902] pci 0000:06:02.5: [8086:154c] type 00 class 0x020000
[   26.493945] pci 0000:06:02.5: Max Payload Size set to 256 (was 128, max 2048)
[   26.494046] iommu: Adding device 0000:06:02.5 to group 101
[   26.494103] pci 0000:06:02.6: [8086:154c] type 00 class 0x020000
[   26.494147] pci 0000:06:02.6: Max Payload Size set to 256 (was 128, max 2048)
[   26.494247] iommu: Adding device 0000:06:02.6 to group 102
[   26.494295] pci 0000:06:02.7: [8086:154c] type 00 class 0x020000
[   26.494339] pci 0000:06:02.7: Max Payload Size set to 256 (was 128, max 2048)
[   26.494446] iommu: Adding device 0000:06:02.7 to group 103
[   26.977585] i40e 0000:06:00.1 enp6s0f1: adding 68:05:ca:37:26:91 vid=0
[   26.985175] i40e 0000:06:00.1: Allocating 8 VFs.
[   27.085659] pci 0000:06:0a.0: [8086:154c] type 00 class 0x020000
[   27.085704] pci 0000:06:0a.0: Max Payload Size set to 256 (was 128, max 2048)
[   27.085868] iommu: Adding device 0000:06:0a.0 to group 104
[   27.085938] pci 0000:06:0a.1: [8086:154c] type 00 class 0x020000
[   27.085977] pci 0000:06:0a.1: Max Payload Size set to 256 (was 128, max 2048)
[   27.086047] iommu: Adding device 0000:06:0a.1 to group 105
[   27.086076] pci 0000:06:0a.2: [8086:154c] type 00 class 0x020000
[   27.086114] pci 0000:06:0a.2: Max Payload Size set to 256 (was 128, max 2048)
[   27.086184] iommu: Adding device 0000:06:0a.2 to group 106
[   27.086213] pci 0000:06:0a.3: [8086:154c] type 00 class 0x020000
[   27.086251] pci 0000:06:0a.3: Max Payload Size set to 256 (was 128, max 2048)
[   27.086321] iommu: Adding device 0000:06:0a.3 to group 107
[   27.086349] pci 0000:06:0a.4: [8086:154c] type 00 class 0x020000
[   27.086386] pci 0000:06:0a.4: Max Payload Size set to 256 (was 128, max 2048)
[   27.086454] iommu: Adding device 0000:06:0a.4 to group 108
[   27.086485] pci 0000:06:0a.5: [8086:154c] type 00 class 0x020000
[   27.086533] pci 0000:06:0a.5: Max Payload Size set to 256 (was 128, max 2048)
[   27.086637] iommu: Adding device 0000:06:0a.5 to group 109
[   27.086674] pci 0000:06:0a.6: [8086:154c] type 00 class 0x020000
[   27.086732] pci 0000:06:0a.6: Max Payload Size set to 256 (was 128, max 2048)
[   27.086829] iommu: Adding device 0000:06:0a.6 to group 110
[   27.086860] pci 0000:06:0a.7: [8086:154c] type 00 class 0x020000
[   27.086908] pci 0000:06:0a.7: Max Payload Size set to 256 (was 128, max 2048)
[   27.087045] iommu: Adding device 0000:06:0a.7 to group 111
[   27.266965] device qg-3e44de10-3d left promiscuous mode
[   27.313043] ip6_tables: (C) 2000-2006 Netfilter Core Team
[   27.545545] i40e 0000:83:00.0 enp131s0f0: adding 68:05:ca:3a:a3:60 vid=0
[   27.558666] i40e 0000:83:00.0: Allocating 8 VFs.
[   27.639645] device qr-912a3637-a9 left promiscuous mode
[   27.659650] pci 0000:83:02.0: [8086:154c] type 00 class 0x020000
[   27.659698] pci 0000:83:02.0: Max Payload Size set to 256 (was 128, max 2048)
[   27.659919] iommu: Adding device 0000:83:02.0 to group 112
[   27.660016] pci 0000:83:02.1: [8086:154c] type 00 class 0x020000
[   27.660055] pci 0000:83:02.1: Max Payload Size set to 256 (was 128, max 2048)
[   27.660146] iommu: Adding device 0000:83:02.1 to group 113
[   27.660191] pci 0000:83:02.2: [8086:154c] type 00 class 0x020000
[   27.660230] pci 0000:83:02.2: Max Payload Size set to 256 (was 128, max 2048)
[   27.660327] iommu: Adding device 0000:83:02.2 to group 114
[   27.660410] pci 0000:83:02.3: [8086:154c] type 00 class 0x020000
[   27.660449] pci 0000:83:02.3: Max Payload Size set to 256 (was 128, max 2048)
[   27.660537] iommu: Adding device 0000:83:02.3 to group 115
[   27.660584] pci 0000:83:02.4: [8086:154c] type 00 class 0x020000
[   27.660634] pci 0000:83:02.4: Max Payload Size set to 256 (was 128, max 2048)
[   27.660725] iommu: Adding device 0000:83:02.4 to group 116
[   27.660771] pci 0000:83:02.5: [8086:154c] type 00 class 0x020000
[   27.660809] pci 0000:83:02.5: Max Payload Size set to 256 (was 128, max 2048)
[   27.660901] iommu: Adding device 0000:83:02.5 to group 117
[   27.660950] pci 0000:83:02.6: [8086:154c] type 00 class 0x020000
[   27.660989] pci 0000:83:02.6: Max Payload Size set to 256 (was 128, max 2048)
[   27.661078] iommu: Adding device 0000:83:02.6 to group 118
[   27.661121] pci 0000:83:02.7: [8086:154c] type 00 class 0x020000
[   27.661160] pci 0000:83:02.7: Max Payload Size set to 256 (was 128, max 2048)
[   27.661246] iommu: Adding device 0000:83:02.7 to group 119
[   27.695063] Ebtables v2.0 registered
[   28.075814] i40e 0000:83:00.1 enp131s0f1: adding 68:05:ca:3a:a3:61 vid=0
[   28.086628] i40e 0000:83:00.1: Allocating 8 VFs.
[   28.108753] device qr-9cc0be6f-14 left promiscuous mode
[   28.187599] pci 0000:83:0a.0: [8086:154c] type 00 class 0x020000
[   28.187641] pci 0000:83:0a.0: Max Payload Size set to 256 (was 128, max 2048)
[   28.187813] iommu: Adding device 0000:83:0a.0 to group 120
[   28.187885] pci 0000:83:0a.1: [8086:154c] type 00 class 0x020000
[   28.187921] pci 0000:83:0a.1: Max Payload Size set to 256 (was 128, max 2048)
[   28.187993] iommu: Adding device 0000:83:0a.1 to group 121
[   28.188028] pci 0000:83:0a.2: [8086:154c] type 00 class 0x020000
[   28.188064] pci 0000:83:0a.2: Max Payload Size set to 256 (was 128, max 2048)
[   28.188136] iommu: Adding device 0000:83:0a.2 to group 122
[   28.188169] pci 0000:83:0a.3: [8086:154c] type 00 class 0x020000
[   28.188205] pci 0000:83:0a.3: Max Payload Size set to 256 (was 128, max 2048)
[   28.188272] iommu: Adding device 0000:83:0a.3 to group 123
[   28.188305] pci 0000:83:0a.4: [8086:154c] type 00 class 0x020000
[   28.188340] pci 0000:83:0a.4: Max Payload Size set to 256 (was 128, max 2048)
[   28.188425] iommu: Adding device 0000:83:0a.4 to group 124
[   28.188468] pci 0000:83:0a.5: [8086:154c] type 00 class 0x020000
[   28.188503] pci 0000:83:0a.5: Max Payload Size set to 256 (was 128, max 2048)
[   28.188620] iommu: Adding device 0000:83:0a.5 to group 125
[   28.188659] pci 0000:83:0a.6: [8086:154c] type 00 class 0x020000
[   28.188694] pci 0000:83:0a.6: Max Payload Size set to 256 (was 128, max 2048)
[   28.188838] iommu: Adding device 0000:83:0a.6 to group 126
[   28.188887] pci 0000:83:0a.7: [8086:154c] type 00 class 0x020000
[   28.188922] pci 0000:83:0a.7: Max Payload Size set to 256 (was 128, max 2048)
[   28.189083] iommu: Adding device 0000:83:0a.7 to group 127
[   28.282306] device qr-e1f33440-0c left promiscuous mode
[   28.360312] device tap502560f9-d8 left promiscuous mode
[   28.396800] device tap9b0fa827-27 left promiscuous mode
[   28.472534] device tapd10cc73d-3b left promiscuous mode
[   29.801012] ixgbe 0000:04:00.0 enp4s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[   41.221646] IPv6: ADDRCONF(NETDEV_UP): qvb7f4dce60-91: link is not ready
[   41.308303] device qvb7f4dce60-91 entered promiscuous mode
[   41.416683] IPv6: ADDRCONF(NETDEV_CHANGE): qvb7f4dce60-91: link becomes ready
[   41.506396] device qvo7f4dce60-91 entered promiscuous mode
[   41.655044] qbr7f4dce60-91: port 1(qvb7f4dce60-91) entered forwarding state
[   41.655051] qbr7f4dce60-91: port 1(qvb7f4dce60-91) entered forwarding state
[   42.000932] Netfilter messages via NETLINK v0.30.
[   42.014636] ip_set: protocol 6
[   89.625075] device tap9b0fa827-27 entered promiscuous mode
[   89.633839] device tap502560f9-d8 entered promiscuous mode
[   89.635991] device tapd10cc73d-3b entered promiscuous mode
[   89.877049] device qr-912a3637-a9 entered promiscuous mode
[   90.138414] device qr-9cc0be6f-14 entered promiscuous mode
[   90.414338] device qr-e1f33440-0c entered promiscuous mode
[   90.702716] device qg-3e44de10-3d entered promiscuous mode
[  980.670259] Unable to load target_core_user
[  994.343776] scsi host7: iSCSI Initiator over TCP/IP
[  994.361276] scsi 7:0:0:0: Direct-Access     LIO-ORG  IBLOCK           4.0  PQ: 0 ANSI: 5
[  994.361705] sd 7:0:0:0: alua: supports implicit and explicit TPGS
[  994.361794] TARGET_CORE[iSCSI]: Expected Transfer Length: 264 does not match SCSI CDB Length: 8 for SAM Opcode: 0x12
[  994.361830] sd 7:0:0:0: alua: No target port descriptors found
[  994.361833] sd 7:0:0:0: alua: Attach failed (-22)
[  994.361862] sd 7:0:0:0: [sdg] 115343360 512-byte logical blocks: (59.0 GB/55.0 GiB)
[  994.361868] sd 7:0:0:0: failed to add device handler: -22
[  994.361911] sd 7:0:0:0: [sdg] Write Protect is off
[  994.361914] sd 7:0:0:0: [sdg] Mode Sense: 00 00 00 00
[  994.361916] sd 7:0:0:0: [sdg] Asking for cache data failed
[  994.361937] sd 7:0:0:0: [sdg] Assuming drive cache: write through
[  994.363181] sd 7:0:0:0: [sdg] Attached SCSI disk
[  994.506461] scsi 7:0:0:0: Direct-Access     LIO-ORG  IBLOCK           4.0  PQ: 0 ANSI: 5
[  994.506822] ------------[ cut here ]------------
[  994.506828] WARNING: CPU: 1 PID: 10460 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x64/0x80()
[  994.506829] sysfs: cannot create duplicate filename '/bus/scsi/devices/7:0:0:0'
[  994.506830] Modules linked in: target_core_pscsi target_core_file target_core_iblock xt_REDIRECT nf_nat_redirect xt_nat xt_mark xt_CHECKSUM nf_conntrack_ipv6 xt_mac xt_physdev xt_set xt_conntrack iptable_raw ip_set_hash_net ip_set nfnetlink veth ebtable_filter ebtables ip6table_filter ip6_tables iscsi_tcp libiscsi_tcp 8021q garp mrp rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod openvswitch bonding ib_srp scsi_transport_srp ib_ipoib nf_defrag_ipv6 rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_sa ib_mad usnic_verbs ib_core xt_comment xt_multiport iptable_filter ib_addr iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 x86_pkg_temp_thermal nf_nat_ipv4 intel_powerclamp nf_nat coretemp kvm_intel nf_conntrack kvm iptable_mangle irqbypass crct10dif_pclmul
[  994.506873]  crc32_pclmul nfsd aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ses enclosure sb_edac sg auth_rpcgss edac_core iTCO_wdt ipmi_devintf iTCO_vendor_support nfs_acl lockd shpchp pcspkr lpc_ich ipmi_si ipmi_msghandler mfd_core wmi acpi_power_meter grace sunrpc br_netfilter bridge stp llc dm_multipath ip_tables xfs libcrc32c sd_mod mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ixgbe(O) drm fnic i40e libfcoe igb megaraid_sas enic libfc crc32c_intel vxlan ip6_udp_tunnel udp_tunnel ptp scsi_transport_fc i2c_algo_bit dca i2c_core pps_core dm_mirror dm_region_hash dm_log dm_mod
[  994.506906] CPU: 1 PID: 10460 Comm: iscsiadm Tainted: G        W  O    4.4.0 #1
[  994.506907] Hardware name: Cisco Systems Inc UCSC-C240-M3S/UCSC-C240-M3S, BIOS C240M3.2.0.8.0.071620152208 07/16/2015
[  994.506909]  0000000000000000 00000000c99ac6bd ffff882d19f07948 ffffffff81329090
[  994.506910]  ffff882d19f07990 ffff882d19f07980 ffffffff810862c6 ffff882d4b39d000
[  994.506912]  ffff882fa039c858 ffff882fa50dee10 0000000000000001 ffffffffffffffef
[  994.506914] Call Trace:
[  994.506920]  [<ffffffff81329090>] dump_stack+0x44/0x64
[  994.506931]  [<ffffffff810862c6>] warn_slowpath_common+0x86/0xc0
[  994.506932]  [<ffffffff8108635c>] warn_slowpath_fmt+0x5c/0x80
[  994.506936]  [<ffffffff8127e9a8>] ? kernfs_path+0x48/0x60
[  994.506937]  [<ffffffff81282014>] sysfs_warn_dup+0x64/0x80
[  994.506939]  [<ffffffff812823aa>] sysfs_do_create_link_sd.isra.2+0xaa/0xb0
[  994.506940]  [<ffffffff812823d5>] sysfs_create_link+0x25/0x40
[  994.506944]  [<ffffffff81454a6b>] bus_add_device+0x10b/0x1f0
[  994.506946]  [<ffffffff81452815>] device_add+0x3b5/0x610
[  994.506949]  [<ffffffff81487df5>] scsi_sysfs_add_sdev+0xa5/0x290
[  994.506951]  [<ffffffff81485355>] scsi_probe_and_add_lun+0xb65/0xd80
[  994.506955]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[  994.506957]  [<ffffffff81485d67>] __scsi_scan_target+0xf7/0x260
[  994.506958]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[  994.506960]  [<ffffffff81485fa7>] scsi_scan_target+0xd7/0xf0
[  994.506967]  [<ffffffffa07529f5>] iscsi_user_scan_session.part.14+0x105/0x140 [scsi_transport_iscsi]
[  994.506971]  [<ffffffffa0752a30>] ? iscsi_user_scan_session.part.14+0x140/0x140 [scsi_transport_iscsi]
[  994.506975]  [<ffffffffa0752a4e>] iscsi_user_scan_session+0x1e/0x30 [scsi_transport_iscsi]
[  994.506976]  [<ffffffff81450b80>] device_for_each_child+0x50/0x90
[  994.506980]  [<ffffffffa075083d>] iscsi_user_scan+0x3d/0x60 [scsi_transport_iscsi]
[  994.506982]  [<ffffffff81487bb6>] store_scan+0xa6/0x100
[  994.506985]  [<ffffffff811e3158>] ? __kmalloc+0x1b8/0x250
[  994.506988]  [<ffffffff81450318>] dev_attr_store+0x18/0x30
[  994.506989]  [<ffffffff8128170a>] sysfs_kf_write+0x3a/0x50
[  994.506991]  [<ffffffff81280d90>] kernfs_fop_write+0x120/0x170
[  994.506993]  [<ffffffff81203f17>] __vfs_write+0x37/0x100
[  994.506995]  [<ffffffff812b6c03>] ? selinux_file_permission+0xc3/0x110
[  994.506999]  [<ffffffff812ae75d>] ? security_file_permission+0x3d/0xc0
[  994.507002]  [<ffffffff810cd95f>] ? percpu_down_read+0x1f/0x50
[  994.507003]  [<ffffffff812045f2>] vfs_write+0xa2/0x1a0
[  994.507006]  [<ffffffff81003176>] ? do_audit_syscall_entry+0x66/0x70
[  994.507008]  [<ffffffff812053b5>] SyS_write+0x55/0xc0
[  994.507011]  [<ffffffff816945ae>] entry_SYSCALL_64_fastpath+0x12/0x71
[  994.507012] ---[ end trace 5d59079faf7d2511 ]---
[  994.507042] scsi 7:0:0:0: failed to add device: -17
[  994.986819] scsi host8: iSCSI Initiator over TCP/IP
[  995.001630] scsi 8:0:0:0: Direct-Access     LIO-ORG  IBLOCK           4.0  PQ: 0 ANSI: 5
[  995.001995] sd 8:0:0:0: alua: supports implicit and explicit TPGS
[  995.002076] TARGET_CORE[iSCSI]: Expected Transfer Length: 264 does not match SCSI CDB Length: 8 for SAM Opcode: 0x12
[  995.002117] sd 8:0:0:0: alua: No target port descriptors found
[  995.002119] sd 8:0:0:0: alua: Attach failed (-22)
[  995.002141] sd 8:0:0:0: [sdh] 2097152 512-byte logical blocks: (1.07 GB/1.00 GiB)
[  995.002142] sd 8:0:0:0: failed to add device handler: -22
[  995.002187] sd 8:0:0:0: [sdh] Write Protect is off
[  995.002189] sd 8:0:0:0: [sdh] Mode Sense: 00 00 00 00
[  995.002191] sd 8:0:0:0: [sdh] Asking for cache data failed
[  995.002211] sd 8:0:0:0: [sdh] Assuming drive cache: write through
[  995.002612] sd 8:0:0:0: [sdh] Attached SCSI disk
[  995.150008] scsi 8:0:0:0: Direct-Access     LIO-ORG  IBLOCK           4.0  PQ: 0 ANSI: 5
[  995.150316] ------------[ cut here ]------------
[  995.150321] WARNING: CPU: 2 PID: 10504 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x64/0x80()
[  995.150322] sysfs: cannot create duplicate filename '/bus/scsi/devices/8:0:0:0'
[  995.150323] Modules linked in: target_core_pscsi target_core_file target_core_iblock xt_REDIRECT nf_nat_redirect xt_nat xt_mark xt_CHECKSUM nf_conntrack_ipv6 xt_mac xt_physdev xt_set xt_conntrack iptable_raw ip_set_hash_net ip_set nfnetlink veth ebtable_filter ebtables ip6table_filter ip6_tables iscsi_tcp libiscsi_tcp 8021q garp mrp rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod openvswitch bonding ib_srp scsi_transport_srp ib_ipoib nf_defrag_ipv6 rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_sa ib_mad usnic_verbs ib_core xt_comment xt_multiport iptable_filter ib_addr iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 x86_pkg_temp_thermal nf_nat_ipv4 intel_powerclamp nf_nat coretemp kvm_intel nf_conntrack kvm iptable_mangle irqbypass crct10dif_pclmul
[  995.150367]  crc32_pclmul nfsd aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ses enclosure sb_edac sg auth_rpcgss edac_core iTCO_wdt ipmi_devintf iTCO_vendor_support nfs_acl lockd shpchp pcspkr lpc_ich ipmi_si ipmi_msghandler mfd_core wmi acpi_power_meter grace sunrpc br_netfilter bridge stp llc dm_multipath ip_tables xfs libcrc32c sd_mod mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ixgbe(O) drm fnic i40e libfcoe igb megaraid_sas enic libfc crc32c_intel vxlan ip6_udp_tunnel udp_tunnel ptp scsi_transport_fc i2c_algo_bit dca i2c_core pps_core dm_mirror dm_region_hash dm_log dm_mod
[  995.150399] CPU: 2 PID: 10504 Comm: iscsiadm Tainted: G        W  O    4.4.0 #1
[  995.150400] Hardware name: Cisco Systems Inc UCSC-C240-M3S/UCSC-C240-M3S, BIOS C240M3.2.0.8.0.071620152208 07/16/2015
[  995.150402]  0000000000000000 0000000000b57f59 ffff882d19ffb948 ffffffff81329090
[  995.150404]  ffff882d19ffb990 ffff882d19ffb980 ffffffff810862c6 ffff88007ac62000
[  995.150405]  ffff882f9f889900 ffff882fa50dee10 0000000000000001 ffffffffffffffef
[  995.150407] Call Trace:
[  995.150412]  [<ffffffff81329090>] dump_stack+0x44/0x64
[  995.150415]  [<ffffffff810862c6>] warn_slowpath_common+0x86/0xc0
[  995.150416]  [<ffffffff8108635c>] warn_slowpath_fmt+0x5c/0x80
[  995.150419]  [<ffffffff8127e9a8>] ? kernfs_path+0x48/0x60
[  995.150420]  [<ffffffff81282014>] sysfs_warn_dup+0x64/0x80
[  995.150435]  [<ffffffff812823aa>] sysfs_do_create_link_sd.isra.2+0xaa/0xb0
[  995.150437]  [<ffffffff812823d5>] sysfs_create_link+0x25/0x40
[  995.150440]  [<ffffffff81454a6b>] bus_add_device+0x10b/0x1f0
[  995.150442]  [<ffffffff81452815>] device_add+0x3b5/0x610
[  995.150445]  [<ffffffff81487df5>] scsi_sysfs_add_sdev+0xa5/0x290
[  995.150447]  [<ffffffff81485355>] scsi_probe_and_add_lun+0xb65/0xd80
[  995.150451]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[  995.150453]  [<ffffffff81485d67>] __scsi_scan_target+0xf7/0x260
[  995.150454]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[  995.150456]  [<ffffffff81485fa7>] scsi_scan_target+0xd7/0xf0
[  995.150463]  [<ffffffffa07529f5>] iscsi_user_scan_session.part.14+0x105/0x140 [scsi_transport_iscsi]
[  995.150467]  [<ffffffffa0752a30>] ? iscsi_user_scan_session.part.14+0x140/0x140 [scsi_transport_iscsi]
[  995.150470]  [<ffffffffa0752a4e>] iscsi_user_scan_session+0x1e/0x30 [scsi_transport_iscsi]
[  995.150472]  [<ffffffff81450b80>] device_for_each_child+0x50/0x90
[  995.150475]  [<ffffffffa075083d>] iscsi_user_scan+0x3d/0x60 [scsi_transport_iscsi]
[  995.150477]  [<ffffffff81487bb6>] store_scan+0xa6/0x100
[  995.150494]  [<ffffffff811e3158>] ? __kmalloc+0x1b8/0x250
[  995.150498]  [<ffffffff81450318>] dev_attr_store+0x18/0x30
[  995.150499]  [<ffffffff8128170a>] sysfs_kf_write+0x3a/0x50
[  995.150500]  [<ffffffff81280d90>] kernfs_fop_write+0x120/0x170
[  995.150503]  [<ffffffff81203f17>] __vfs_write+0x37/0x100
[  995.150505]  [<ffffffff812b6c03>] ? selinux_file_permission+0xc3/0x110
[  995.150509]  [<ffffffff812ae75d>] ? security_file_permission+0x3d/0xc0
[  995.150512]  [<ffffffff810cd95f>] ? percpu_down_read+0x1f/0x50
[  995.150513]  [<ffffffff812045f2>] vfs_write+0xa2/0x1a0
[  995.150518]  [<ffffffff81003176>] ? do_audit_syscall_entry+0x66/0x70
[  995.150520]  [<ffffffff812053b5>] SyS_write+0x55/0xc0
[  995.150523]  [<ffffffff816945ae>] entry_SYSCALL_64_fastpath+0x12/0x71
[  995.150525] ---[ end trace 5d59079faf7d2512 ]---
[  995.150556] scsi 8:0:0:0: failed to add device: -17

with kernel 4.4.0

lsscsi 
[0:0:44:0]   enclosu CISCO    UCS 240          0809  -        
[0:2:0:0]    disk    LSI      MR9261-8i        2.13  /dev/sda 
[0:2:1:0]    disk    LSI      MR9261-8i        2.13  /dev/sdb 
[3:0:0:2]    type?   vendor?  model?           rev?  -        
[4:0:0:3]    type?   vendor?  model?           rev?  -        
[5:0:0:1]    type?   vendor?  model?           rev?  -        
[6:0:0:4]    type?   vendor?  model?           rev?  -        
[7:0:0:0]    type?   vendor?  model?           rev?  -        
[8:0:0:0]    type?   vendor?  model?           rev?  -    

with kernel 3.10.0-327.4.4

lsscsi 
[0:0:44:0]   enclosu CISCO    UCS 240          0809  -        
[0:2:0:0]    disk    LSI      MR9261-8i        2.13  /dev/sda 
[0:2:1:0]    disk    LSI      MR9261-8i        2.13  /dev/sdb 
[3:0:0:2]    disk    LIO-ORG  san-disk-2       4.0   /dev/sdc 
[4:0:0:3]    disk    LIO-ORG  san-disk-3       4.0   /dev/sdd 
[5:0:0:1]    disk    LIO-ORG  san-disk-1       4.0   /dev/sde 
[6:0:0:4]    disk    LIO-ORG  san-disk-4       4.0   /dev/sdf

Actually this server uses both cases: Local taregts (since it is OpenStack server) and remote targets as it tries to mount 4 remotefile systems.  

You are correct, I always use the same box I just change the kernel it is using to boot. No other changes to the environment. I do not mind to load a test kernel without that suspected patch, just get me the RPM.

As you can see the first type of traceback happens when the server tries to attach to remote targets, the second type of traceback happens when I try to start OpenStack instance with local LVM volume which is represented as a local iscsi target. Please see below.

[Remote taregt case]

targetcli
targetcli shell version 2.1.fb41
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.

/> ls
o- / ......................................................................................................................... [...]
  o- backstores .............................................................................................................. [...]
  | o- block .................................................................................................. [Storage Objects: 0]
  | o- fileio ................................................................................................. [Storage Objects: 0]
  | o- pscsi .................................................................................................. [Storage Objects: 0]
  | o- ramdisk ................................................................................................ [Storage Objects: 0]
  o- iscsi ............................................................................................................ [Targets: 0]
  o- loopback ......................................................................................................... [Targets: 0]
/> exit


[   25.670239] scsi host3: iSCSI Initiator over TCP/IP
[   25.682122] scsi host4: iSCSI Initiator over TCP/IP
[   25.692401] scsi host5: iSCSI Initiator over TCP/IP
[   25.693530] scsi 3:0:0:2: Direct-Access     LIO-ORG  san-disk-2       4.0  PQ: 0 ANSI: 5     < ------------------------ Remote target
[   25.693936] sd 3:0:0:2: alua: supports implicit and explicit TPGS
[   25.694078] sd 3:0:0:2: alua: No target port descriptors found
[   25.694080] sd 3:0:0:2: alua: Attach failed (-22)
[   25.694081] sd 3:0:0:2: failed to add device handler: -22
[   25.694121] sd 3:0:0:2: [sdc] 20507809792 512-byte logical blocks: (10.4 TB/9.54 TiB)
[   25.694127] sd 3:0:0:2: [sdc] Write Protect is off
[   25.694129] sd 3:0:0:2: [sdc] Mode Sense: 00 00 00 00
[   25.694131] sd 3:0:0:2: [sdc] Asking for cache data failed
[   25.694132] sd 3:0:0:2: [sdc] Assuming drive cache: write through
[   25.695112] ------------[ cut here ]------------
[   25.695119] WARNING: CPU: 12 PID: 841 at block/genhd.c:626 add_disk+0x443/0x4a0()
[   25.695120] Modules linked in: iscsi_tcp libiscsi_tcp 8021q garp mrp rpcrdma ib_isert iscsi_target_mod bonding ib_iser libiscsi scsi_transport_iscsi openvswitch ib_srpt target_core_mod nf_defrag_ipv6 ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm xt_comment xt_multiport ib_sa iptable_filter ib_mad iptable_nat nf_conntrack_ipv4 usnic_verbs nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ib_core ib_addr iptable_mangle x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul aesni_intel nfsd lrw gf128mul glue_helper ablk_helper ses cryptd enclosure ipmi_devintf sg sb_edac iTCO_wdt shpchp edac_core iTCO_vendor_support ipmi_si pcspkr lpc_ich ipmi_msghandler mfd_core auth_rpcgss acpi_power_meter wmi nfs_acl lockd br_netfilter
[   25.695161]  grace sunrpc bridge stp llc dm_multipath ip_tables xfs libcrc32c sd_mod mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ixgbe(O) fnic i40e libfcoe igb crc32c_intel megaraid_sas enic libfc vxlan ip6_udp_tunnel udp_tunnel ptp scsi_transport_fc i2c_algo_bit i2c_core pps_core dca dm_mirror dm_region_hash dm_log dm_mod
[   25.695183] CPU: 12 PID: 841 Comm: kworker/u82:2 Tainted: G           O    4.4.0 #1
[   25.695185] Hardware name: Cisco Systems Inc UCSC-C240-M3S/UCSC-C240-M3S, BIOS C240M3.2.0.8.0.071620152208 07/16/2015
[   25.695189] Workqueue: events_unbound async_run_entry_fn
[   25.695191]  0000000000000000 000000004b5682c8 ffff885f9b887cf8 ffffffff81329090
[   25.695193]  0000000000000000 ffff885f9b887d30 ffffffff810862c6 ffff885fa5ef2000
[   25.695194]  ffff885fa5ef2080 ffff885f9d2fe1a0 ffff885fa5ef200c ffff885f86b10000
[   25.695196] Call Trace:
[   25.695204]  [<ffffffff81329090>] dump_stack+0x44/0x64
[   25.695207]  [<ffffffff810862c6>] warn_slowpath_common+0x86/0xc0
[   25.695209]  [<ffffffff8108640a>] warn_slowpath_null+0x1a/0x20
[   25.695210]  [<ffffffff8130ef23>] add_disk+0x443/0x4a0
[   25.695216]  [<ffffffffa008ba35>] sd_probe_async+0x115/0x1d0 [sd_mod]
[   25.695217]  [<ffffffff810a70fa>] async_run_entry_fn+0x4a/0x140
[   25.695220]  [<ffffffff8109e09c>] process_one_work+0x14c/0x3c0
[   25.695221]  [<ffffffff8109e8d4>] worker_thread+0x114/0x470
[   25.695226]  [<ffffffff816905ba>] ? __schedule+0x2aa/0x700
[   25.695237]  [<ffffffff8109e7c0>] ? rescuer_thread+0x310/0x310
[   25.695241]  [<ffffffff810a43d8>] kthread+0xd8/0xf0
[   25.695243]  [<ffffffff810a4300>] ? kthread_park+0x60/0x60
[   25.695245]  [<ffffffff8169490f>] ret_from_fork+0x3f/0x70
[   25.695247]  [<ffffffff810a4300>] ? kthread_park+0x60/0x60
[   25.695248] ---[ end trace bcfb1475313b5f60 ]---
[   25.695262] sd 3:0:0:2: [sdc] Attached SCSI disk
[   25.703053] scsi host6: iSCSI Initiator over TCP/IP
[   25.704370] scsi 3:0:0:0: Unexpected response from lun 2 while scanning, scan aborted


[local target case]


/> ls
o- / ......................................................................................................................... [...]
  o- backstores .............................................................................................................. [...]
  | o- block .................................................................................................. [Storage Objects: 1]
  | | o- iqn.2010-10.org.openstack:volume-07e39fbc-4767-4455-bb96-c6b53be6a10c  [/dev/cinder-volumes-local/volume-07e39fbc-4767-4455-bb96-c6b53be6a10c (55.0GiB) write-thru activated]
  | o- fileio ................................................................................................. [Storage Objects: 0]
  | o- pscsi .................................................................................................. [Storage Objects: 0]
  | o- ramdisk ................................................................................................ [Storage Objects: 0]
  o- iscsi ............................................................................................................ [Targets: 1]
  | o- iqn.2010-10.org.openstack:volume-07e39fbc-4767-4455-bb96-c6b53be6a10c ............................................. [TPGs: 1]
  |   o- tpg1 .......................................................................................... [no-gen-acls, auth per-acl]
  |     o- acls .......................................................................................................... [ACLs: 1]
  |     | o- iqn.1994-05.com.redhat:cf7f1fafca4b ...................................................... [1-way auth, Mapped LUNs: 1]
  |     |   o- mapped_lun0 ................. [lun0 block/iqn.2010-10.org.openstack:volume-07e39fbc-4767-4455-bb96-c6b53be6a10c (rw)]
  |     o- luns .......................................................................................................... [LUNs: 1]
  |     | o- lun0  [block/iqn.2010-10.org.openstack:volume-07e39fbc-4767-4455-bb96-c6b53be6a10c (/dev/cinder-volumes-local/volume-07e39fbc-4767-4455-bb96-c6b53be6a10c)]
  |     o- portals .................................................................................................... [Portals: 1]
  |       o- 192.168.80.23:3260 ............................................................................................... [OK]
  o- loopback ......................................................................................................... [Targets: 0]
/>


[root@sbezverk-osp-3 ~(keystone_admin)]# lvs
  LV                                          VG                   Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  swift-volume                                cinder-volumes-local -wi-ao---- 200.00g                                                    
  volume-07e39fbc-4767-4455-bb96-c6b53be6a10c cinder-volumes-local -wi-a-----  55.00g              < ----- Local volume and a local target for OpenStack instance                                      
  volume-c69da50c-c66a-4f94-9a40-b4f7d7f28eae cinder-volumes-local -wi-a-----   1.00g                    < ----- Local volume and a local target for OpenStack instance           


Here are tracebacks when OpenStack tries to attach to these two local targets:

[ 1019.953129] Unable to load target_core_user
[ 1033.758866] scsi host7: iSCSI Initiator over TCP/IP
[ 1033.776279] scsi 7:0:0:0: Direct-Access     LIO-ORG  IBLOCK           4.0  PQ: 0 ANSI: 5
[ 1033.776658] sd 7:0:0:0: alua: supports implicit and explicit TPGS
[ 1033.776721] TARGET_CORE[iSCSI]: Expected Transfer Length: 264 does not match SCSI CDB Length: 8 for SAM Opcode: 0x12
[ 1033.776763] sd 7:0:0:0: alua: No target port descriptors found
[ 1033.776765] sd 7:0:0:0: alua: Attach failed (-22)
[ 1033.776782] sd 7:0:0:0: failed to add device handler: -22
[ 1033.776786] sd 7:0:0:0: [sdg] 115343360 512-byte logical blocks: (59.0 GB/55.0 GiB)             < ----------------------------------- first 55G volume
[ 1033.776791] sd 7:0:0:0: [sdg] Write Protect is off
[ 1033.776792] sd 7:0:0:0: [sdg] Mode Sense: 00 00 00 00
[ 1033.776795] sd 7:0:0:0: [sdg] Asking for cache data failed
[ 1033.776820] sd 7:0:0:0: [sdg] Assuming drive cache: write through
[ 1033.776960] ------------[ cut here ]------------
[ 1033.776965] WARNING: CPU: 6 PID: 7 at lib/kobject.c:244 kobject_add_internal+0x10d/0x320()
[ 1033.776966] kobject_add_internal failed for block (error: -2 parent: 7:0:0:0)
[ 1033.776967] Modules linked in: target_core_pscsi target_core_file target_core_iblock xt_REDIRECT nf_nat_redirect xt_nat xt_mark xt_CHECKSUM nf_conntrack_ipv6 xt_mac xt_physdev xt_set xt_conntrack iptable_raw ip_set_hash_net ip_set nfnetlink veth ebtable_filter ebtables ip6table_filter ip6_tables iscsi_tcp libiscsi_tcp 8021q garp mrp rpcrdma ib_isert iscsi_target_mod bonding ib_iser libiscsi scsi_transport_iscsi openvswitch ib_srpt target_core_mod nf_defrag_ipv6 ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm xt_comment xt_multiport ib_sa iptable_filter ib_mad iptable_nat nf_conntrack_ipv4 usnic_verbs nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ib_core ib_addr iptable_mangle x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul
[ 1033.777008]  crc32_pclmul aesni_intel nfsd lrw gf128mul glue_helper ablk_helper ses cryptd enclosure ipmi_devintf sg sb_edac iTCO_wdt shpchp edac_core iTCO_vendor_support ipmi_si pcspkr lpc_ich ipmi_msghandler mfd_core auth_rpcgss acpi_power_meter wmi nfs_acl lockd br_netfilter grace sunrpc bridge stp llc dm_multipath ip_tables xfs libcrc32c sd_mod mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ixgbe(O) fnic i40e libfcoe igb crc32c_intel megaraid_sas enic libfc vxlan ip6_udp_tunnel udp_tunnel ptp scsi_transport_fc i2c_algo_bit i2c_core pps_core dca dm_mirror dm_region_hash dm_log dm_mod
[ 1033.777040] CPU: 6 PID: 7 Comm: kworker/u81:0 Tainted: G        W  O    4.4.0 #1
[ 1033.777041] Hardware name: Cisco Systems Inc UCSC-C240-M3S/UCSC-C240-M3S, BIOS C240M3.2.0.8.0.071620152208 07/16/2015
[ 1033.777045] Workqueue: events_unbound async_run_entry_fn
[ 1033.777046]  0000000000000000 00000000f9d0f617 ffff882fa6bcbb60 ffffffff81329090
[ 1033.777048]  ffff882fa6bcbba8 ffff882fa6bcbb98 ffffffff810862c6 ffff885d991d0360
[ 1033.777050]  ffff885fa5f5c978 00000000fffffffe ffff885fa5f5c978 ffff882e7b520070
[ 1033.777052] Call Trace:
[ 1033.777057]  [<ffffffff81329090>] dump_stack+0x44/0x64
[ 1033.777060]  [<ffffffff810862c6>] warn_slowpath_common+0x86/0xc0
[ 1033.777061]  [<ffffffff8108635c>] warn_slowpath_fmt+0x5c/0x80
[ 1033.777063]  [<ffffffff8132bebd>] kobject_add_internal+0x10d/0x320
[ 1033.777067]  [<ffffffff811a34d2>] ? kfree_const+0x22/0x30
[ 1033.777068]  [<ffffffff8132c305>] kobject_add+0x75/0xd0
[ 1033.777073]  [<ffffffff811e2a8e>] ? kmem_cache_alloc_trace+0x1be/0x1e0
[ 1033.777076]  [<ffffffff81451f62>] ? get_device_parent+0x122/0x1d0
[ 1033.777078]  [<ffffffff81451fa8>] get_device_parent+0x168/0x1d0
[ 1033.777079]  [<ffffffff81452557>] device_add+0xf7/0x610
[ 1033.777081]  [<ffffffff8130ecad>] add_disk+0x1cd/0x4a0
[ 1033.777085]  [<ffffffff81460ec5>] ? update_autosuspend+0x55/0x60
[ 1033.777088]  [<ffffffff81460f84>] ? __pm_runtime_use_autosuspend+0x54/0x70
[ 1033.777092]  [<ffffffffa008ba35>] sd_probe_async+0x115/0x1d0 [sd_mod]
[ 1033.777093]  [<ffffffff810a70fa>] async_run_entry_fn+0x4a/0x140
[ 1033.777096]  [<ffffffff8109e09c>] process_one_work+0x14c/0x3c0
[ 1033.777097]  [<ffffffff8109e8d4>] worker_thread+0x114/0x470
[ 1033.777098]  [<ffffffff8109e7c0>] ? rescuer_thread+0x310/0x310
[ 1033.777101]  [<ffffffff810a43d8>] kthread+0xd8/0xf0
[ 1033.777103]  [<ffffffff810a4300>] ? kthread_park+0x60/0x60
[ 1033.777105]  [<ffffffff8169490f>] ret_from_fork+0x3f/0x70
[ 1033.777107]  [<ffffffff810a4300>] ? kthread_park+0x60/0x60
[ 1033.777108] ---[ end trace bcfb1475313b5f61 ]---
[ 1033.777855] sd 7:0:0:0: [sdg] Attached SCSI disk
[ 1033.911169] scsi 7:0:0:0: Direct-Access     LIO-ORG  IBLOCK           4.0  PQ: 0 ANSI: 5
[ 1033.911465] ------------[ cut here ]------------
[ 1033.911470] WARNING: CPU: 22 PID: 10565 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x64/0x80()
[ 1033.911471] sysfs: cannot create duplicate filename '/bus/scsi/devices/7:0:0:0'
[ 1033.911472] Modules linked in: target_core_pscsi target_core_file target_core_iblock xt_REDIRECT nf_nat_redirect xt_nat xt_mark xt_CHECKSUM nf_conntrack_ipv6 xt_mac xt_physdev xt_set xt_conntrack iptable_raw ip_set_hash_net ip_set nfnetlink veth ebtable_filter ebtables ip6table_filter ip6_tables iscsi_tcp libiscsi_tcp 8021q garp mrp rpcrdma ib_isert iscsi_target_mod bonding ib_iser libiscsi scsi_transport_iscsi openvswitch ib_srpt target_core_mod nf_defrag_ipv6 ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm xt_comment xt_multiport ib_sa iptable_filter ib_mad iptable_nat nf_conntrack_ipv4 usnic_verbs nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ib_core ib_addr iptable_mangle x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul
[ 1033.911502]  crc32_pclmul aesni_intel nfsd lrw gf128mul glue_helper ablk_helper ses cryptd enclosure ipmi_devintf sg sb_edac iTCO_wdt shpchp edac_core iTCO_vendor_support ipmi_si pcspkr lpc_ich ipmi_msghandler mfd_core auth_rpcgss acpi_power_meter wmi nfs_acl lockd br_netfilter grace sunrpc bridge stp llc dm_multipath ip_tables xfs libcrc32c sd_mod mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ixgbe(O) fnic i40e libfcoe igb crc32c_intel megaraid_sas enic libfc vxlan ip6_udp_tunnel udp_tunnel ptp scsi_transport_fc i2c_algo_bit i2c_core pps_core dca dm_mirror dm_region_hash dm_log dm_mod
[ 1033.911533] CPU: 22 PID: 10565 Comm: iscsiadm Tainted: G        W  O    4.4.0 #1
[ 1033.911534] Hardware name: Cisco Systems Inc UCSC-C240-M3S/UCSC-C240-M3S, BIOS C240M3.2.0.8.0.071620152208 07/16/2015
[ 1033.911535]  0000000000000000 000000004e211399 ffff885d9a9d3948 ffffffff81329090
[ 1033.911537]  ffff885d9a9d3990 ffff885d9a9d3980 ffffffff810862c6 ffff882fa033c000
[ 1033.911538]  ffff885f9f5bf490 ffff885fa5329e10 0000000000000001 ffffffffffffffef
[ 1033.911540] Call Trace:
[ 1033.911545]  [<ffffffff81329090>] dump_stack+0x44/0x64
[ 1033.911548]  [<ffffffff810862c6>] warn_slowpath_common+0x86/0xc0
[ 1033.911549]  [<ffffffff8108635c>] warn_slowpath_fmt+0x5c/0x80
[ 1033.911552]  [<ffffffff8127e9a8>] ? kernfs_path+0x48/0x60
[ 1033.911553]  [<ffffffff81282014>] sysfs_warn_dup+0x64/0x80
[ 1033.911555]  [<ffffffff812823aa>] sysfs_do_create_link_sd.isra.2+0xaa/0xb0
[ 1033.911556]  [<ffffffff812823d5>] sysfs_create_link+0x25/0x40
[ 1033.911560]  [<ffffffff81454a6b>] bus_add_device+0x10b/0x1f0
[ 1033.911561]  [<ffffffff81452815>] device_add+0x3b5/0x610
[ 1033.911564]  [<ffffffff81487df5>] scsi_sysfs_add_sdev+0xa5/0x290
[ 1033.911566]  [<ffffffff81485355>] scsi_probe_and_add_lun+0xb65/0xd80
[ 1033.911570]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[ 1033.911572]  [<ffffffff81485d67>] __scsi_scan_target+0xf7/0x260
[ 1033.911574]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[ 1033.911577]  [<ffffffff81485fa7>] scsi_scan_target+0xd7/0xf0
[ 1033.911587]  [<ffffffffa07c29f5>] iscsi_user_scan_session.part.14+0x105/0x140 [scsi_transport_iscsi]
[ 1033.911592]  [<ffffffffa07c2a30>] ? iscsi_user_scan_session.part.14+0x140/0x140 [scsi_transport_iscsi]
[ 1033.911596]  [<ffffffffa07c2a4e>] iscsi_user_scan_session+0x1e/0x30 [scsi_transport_iscsi]
[ 1033.911599]  [<ffffffff81450b80>] device_for_each_child+0x50/0x90
[ 1033.911604]  [<ffffffffa07c083d>] iscsi_user_scan+0x3d/0x60 [scsi_transport_iscsi]
[ 1033.911607]  [<ffffffff81487bb6>] store_scan+0xa6/0x100
[ 1033.911612]  [<ffffffff811e3158>] ? __kmalloc+0x1b8/0x250
[ 1033.911616]  [<ffffffff81450318>] dev_attr_store+0x18/0x30
[ 1033.911618]  [<ffffffff8128170a>] sysfs_kf_write+0x3a/0x50
[ 1033.911620]  [<ffffffff81280d90>] kernfs_fop_write+0x120/0x170
[ 1033.911623]  [<ffffffff81203f17>] __vfs_write+0x37/0x100
[ 1033.911626]  [<ffffffff812b6c03>] ? selinux_file_permission+0xc3/0x110
[ 1033.911630]  [<ffffffff812ae75d>] ? security_file_permission+0x3d/0xc0
[ 1033.911634]  [<ffffffff810cd95f>] ? percpu_down_read+0x1f/0x50
[ 1033.911636]  [<ffffffff812045f2>] vfs_write+0xa2/0x1a0
[ 1033.911640]  [<ffffffff81003176>] ? do_audit_syscall_entry+0x66/0x70
[ 1033.911641]  [<ffffffff812053b5>] SyS_write+0x55/0xc0
[ 1033.911644]  [<ffffffff816945ae>] entry_SYSCALL_64_fastpath+0x12/0x71
[ 1033.911645] ---[ end trace bcfb1475313b5f62 ]---
[ 1033.911676] scsi 7:0:0:0: failed to add device: -17
[ 1033.982858] scsi 7:0:0:0: Direct-Access     LIO-ORG  IBLOCK           4.0  PQ: 0 ANSI: 5
[ 1033.983185] ------------[ cut here ]------------
[ 1033.983191] WARNING: CPU: 1 PID: 10574 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x64/0x80()
[ 1033.983192] sysfs: cannot create duplicate filename '/bus/scsi/devices/7:0:0:0'
[ 1033.983193] Modules linked in: target_core_pscsi target_core_file target_core_iblock xt_REDIRECT nf_nat_redirect xt_nat xt_mark xt_CHECKSUM nf_conntrack_ipv6 xt_mac xt_physdev xt_set xt_conntrack iptable_raw ip_set_hash_net ip_set nfnetlink veth ebtable_filter ebtables ip6table_filter ip6_tables iscsi_tcp libiscsi_tcp 8021q garp mrp rpcrdma ib_isert iscsi_target_mod bonding ib_iser libiscsi scsi_transport_iscsi openvswitch ib_srpt target_core_mod nf_defrag_ipv6 ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm xt_comment xt_multiport ib_sa iptable_filter ib_mad iptable_nat nf_conntrack_ipv4 usnic_verbs nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ib_core ib_addr iptable_mangle x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul
[ 1033.983224]  crc32_pclmul aesni_intel nfsd lrw gf128mul glue_helper ablk_helper ses cryptd enclosure ipmi_devintf sg sb_edac iTCO_wdt shpchp edac_core iTCO_vendor_support ipmi_si pcspkr lpc_ich ipmi_msghandler mfd_core auth_rpcgss acpi_power_meter wmi nfs_acl lockd br_netfilter grace sunrpc bridge stp llc dm_multipath ip_tables xfs libcrc32c sd_mod mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ixgbe(O) fnic i40e libfcoe igb crc32c_intel megaraid_sas enic libfc vxlan ip6_udp_tunnel udp_tunnel ptp scsi_transport_fc i2c_algo_bit i2c_core pps_core dca dm_mirror dm_region_hash dm_log dm_mod
[ 1033.983255] CPU: 1 PID: 10574 Comm: iscsiadm Tainted: G        W  O    4.4.0 #1
[ 1033.983256] Hardware name: Cisco Systems Inc UCSC-C240-M3S/UCSC-C240-M3S, BIOS C240M3.2.0.8.0.071620152208 07/16/2015
[ 1033.983258]  0000000000000000 00000000b4fefb6b ffff882e7b617948 ffffffff81329090
[ 1033.983260]  ffff882e7b617990 ffff882e7b617980 ffffffff810862c6 ffff882e793b5000
[ 1033.983261]  ffff882f9f78ded0 ffff885fa5329e10 0000000000000001 ffffffffffffffef
[ 1033.983263] Call Trace:
[ 1033.983268]  [<ffffffff81329090>] dump_stack+0x44/0x64
[ 1033.983270]  [<ffffffff810862c6>] warn_slowpath_common+0x86/0xc0
[ 1033.983271]  [<ffffffff8108635c>] warn_slowpath_fmt+0x5c/0x80
[ 1033.983274]  [<ffffffff8127e9a8>] ? kernfs_path+0x48/0x60
[ 1033.983276]  [<ffffffff81282014>] sysfs_warn_dup+0x64/0x80
[ 1033.983278]  [<ffffffff812823aa>] sysfs_do_create_link_sd.isra.2+0xaa/0xb0
[ 1033.983279]  [<ffffffff812823d5>] sysfs_create_link+0x25/0x40
[ 1033.983282]  [<ffffffff81454a6b>] bus_add_device+0x10b/0x1f0
[ 1033.983284]  [<ffffffff81452815>] device_add+0x3b5/0x610
[ 1033.983287]  [<ffffffff81487df5>] scsi_sysfs_add_sdev+0xa5/0x290
[ 1033.983289]  [<ffffffff81485355>] scsi_probe_and_add_lun+0xb65/0xd80
[ 1033.983293]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[ 1033.983295]  [<ffffffff81485d67>] __scsi_scan_target+0xf7/0x260
[ 1033.983297]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[ 1033.983298]  [<ffffffff81485fa7>] scsi_scan_target+0xd7/0xf0
[ 1033.983306]  [<ffffffffa07c29f5>] iscsi_user_scan_session.part.14+0x105/0x140 [scsi_transport_iscsi]
[ 1033.983309]  [<ffffffffa07c2a30>] ? iscsi_user_scan_session.part.14+0x140/0x140 [scsi_transport_iscsi]
[ 1033.983313]  [<ffffffffa07c2a4e>] iscsi_user_scan_session+0x1e/0x30 [scsi_transport_iscsi]
[ 1033.983314]  [<ffffffff81450b80>] device_for_each_child+0x50/0x90
[ 1033.983318]  [<ffffffffa07c083d>] iscsi_user_scan+0x3d/0x60 [scsi_transport_iscsi]
[ 1033.983320]  [<ffffffff81487bb6>] store_scan+0xa6/0x100
[ 1033.983324]  [<ffffffff811e3158>] ? __kmalloc+0x1b8/0x250
[ 1033.983327]  [<ffffffff81450318>] dev_attr_store+0x18/0x30
[ 1033.983329]  [<ffffffff8128170a>] sysfs_kf_write+0x3a/0x50
[ 1033.983330]  [<ffffffff81280d90>] kernfs_fop_write+0x120/0x170
[ 1033.983332]  [<ffffffff81203f17>] __vfs_write+0x37/0x100
[ 1033.983335]  [<ffffffff812b6c03>] ? selinux_file_permission+0xc3/0x110
[ 1033.983338]  [<ffffffff812ae75d>] ? security_file_permission+0x3d/0xc0
[ 1033.983341]  [<ffffffff810cd95f>] ? percpu_down_read+0x1f/0x50
[ 1033.983342]  [<ffffffff812045f2>] vfs_write+0xa2/0x1a0
[ 1033.983345]  [<ffffffff81003176>] ? do_audit_syscall_entry+0x66/0x70
[ 1033.983346]  [<ffffffff812053b5>] SyS_write+0x55/0xc0
[ 1033.983349]  [<ffffffff816945ae>] entry_SYSCALL_64_fastpath+0x12/0x71
[ 1033.983350] ---[ end trace bcfb1475313b5f63 ]---
[ 1033.983379] scsi 7:0:0:0: failed to add device: -17
[ 1035.052654] scsi 7:0:0:0: Direct-Access     LIO-ORG  IBLOCK           4.0  PQ: 0 ANSI: 5
[ 1035.052938] ------------[ cut here ]------------
[ 1035.052943] WARNING: CPU: 1 PID: 10583 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x64/0x80()
[ 1035.052944] sysfs: cannot create duplicate filename '/bus/scsi/devices/7:0:0:0'
[ 1035.052945] Modules linked in: target_core_pscsi target_core_file target_core_iblock xt_REDIRECT nf_nat_redirect xt_nat xt_mark xt_CHECKSUM nf_conntrack_ipv6 xt_mac xt_physdev xt_set xt_conntrack iptable_raw ip_set_hash_net ip_set nfnetlink veth ebtable_filter ebtables ip6table_filter ip6_tables iscsi_tcp libiscsi_tcp 8021q garp mrp rpcrdma ib_isert iscsi_target_mod bonding ib_iser libiscsi scsi_transport_iscsi openvswitch ib_srpt target_core_mod nf_defrag_ipv6 ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm xt_comment xt_multiport ib_sa iptable_filter ib_mad iptable_nat nf_conntrack_ipv4 usnic_verbs nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ib_core ib_addr iptable_mangle x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul
[ 1035.052976]  crc32_pclmul aesni_intel nfsd lrw gf128mul glue_helper ablk_helper ses cryptd enclosure ipmi_devintf sg sb_edac iTCO_wdt shpchp edac_core iTCO_vendor_support ipmi_si pcspkr lpc_ich ipmi_msghandler mfd_core auth_rpcgss acpi_power_meter wmi nfs_acl lockd br_netfilter grace sunrpc bridge stp llc dm_multipath ip_tables xfs libcrc32c sd_mod mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ixgbe(O) fnic i40e libfcoe igb crc32c_intel megaraid_sas enic libfc vxlan ip6_udp_tunnel udp_tunnel ptp scsi_transport_fc i2c_algo_bit i2c_core pps_core dca dm_mirror dm_region_hash dm_log dm_mod
[ 1035.053043] CPU: 1 PID: 10583 Comm: iscsiadm Tainted: G        W  O    4.4.0 #1
[ 1035.053044] Hardware name: Cisco Systems Inc UCSC-C240-M3S/UCSC-C240-M3S, BIOS C240M3.2.0.8.0.071620152208 07/16/2015
[ 1035.053045]  0000000000000000 00000000acef2a8b ffff882e7b603948 ffffffff81329090
[ 1035.053047]  ffff882e7b603990 ffff882e7b603980 ffffffff810862c6 ffff882e793b5000
[ 1035.053049]  ffff882f9feaa880 ffff885fa5329e10 0000000000000001 ffffffffffffffef
[ 1035.053050] Call Trace:
[ 1035.053056]  [<ffffffff81329090>] dump_stack+0x44/0x64
[ 1035.053058]  [<ffffffff810862c6>] warn_slowpath_common+0x86/0xc0
[ 1035.053060]  [<ffffffff8108635c>] warn_slowpath_fmt+0x5c/0x80
[ 1035.053063]  [<ffffffff8127e9a8>] ? kernfs_path+0x48/0x60
[ 1035.053064]  [<ffffffff81282014>] sysfs_warn_dup+0x64/0x80
[ 1035.053066]  [<ffffffff812823aa>] sysfs_do_create_link_sd.isra.2+0xaa/0xb0
[ 1035.053068]  [<ffffffff812823d5>] sysfs_create_link+0x25/0x40
[ 1035.053071]  [<ffffffff81454a6b>] bus_add_device+0x10b/0x1f0
[ 1035.053073]  [<ffffffff81452815>] device_add+0x3b5/0x610
[ 1035.053076]  [<ffffffff81487df5>] scsi_sysfs_add_sdev+0xa5/0x290
[ 1035.053078]  [<ffffffff81485355>] scsi_probe_and_add_lun+0xb65/0xd80
[ 1035.053082]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[ 1035.053084]  [<ffffffff81485d67>] __scsi_scan_target+0xf7/0x260
[ 1035.053085]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[ 1035.053087]  [<ffffffff81485fa7>] scsi_scan_target+0xd7/0xf0
[ 1035.053094]  [<ffffffffa07c29f5>] iscsi_user_scan_session.part.14+0x105/0x140 [scsi_transport_iscsi]
[ 1035.053097]  [<ffffffffa07c2a30>] ? iscsi_user_scan_session.part.14+0x140/0x140 [scsi_transport_iscsi]
[ 1035.053101]  [<ffffffffa07c2a4e>] iscsi_user_scan_session+0x1e/0x30 [scsi_transport_iscsi]
[ 1035.053102]  [<ffffffff81450b80>] device_for_each_child+0x50/0x90
[ 1035.053106]  [<ffffffffa07c083d>] iscsi_user_scan+0x3d/0x60 [scsi_transport_iscsi]
[ 1035.053108]  [<ffffffff81487bb6>] store_scan+0xa6/0x100
[ 1035.053111]  [<ffffffff811e3158>] ? __kmalloc+0x1b8/0x250
[ 1035.053115]  [<ffffffff81450318>] dev_attr_store+0x18/0x30
[ 1035.053116]  [<ffffffff8128170a>] sysfs_kf_write+0x3a/0x50
[ 1035.053118]  [<ffffffff81280d90>] kernfs_fop_write+0x120/0x170
[ 1035.053120]  [<ffffffff81203f17>] __vfs_write+0x37/0x100
[ 1035.053122]  [<ffffffff812b6c03>] ? selinux_file_permission+0xc3/0x110
[ 1035.053126]  [<ffffffff812ae75d>] ? security_file_permission+0x3d/0xc0
[ 1035.053129]  [<ffffffff810cd95f>] ? percpu_down_read+0x1f/0x50
[ 1035.053130]  [<ffffffff812045f2>] vfs_write+0xa2/0x1a0
[ 1035.053132]  [<ffffffff81003176>] ? do_audit_syscall_entry+0x66/0x70
[ 1035.053134]  [<ffffffff812053b5>] SyS_write+0x55/0xc0
[ 1035.053137]  [<ffffffff816945ae>] entry_SYSCALL_64_fastpath+0x12/0x71
[ 1035.053138] ---[ end trace bcfb1475313b5f64 ]---
[ 1035.053185] scsi 7:0:0:0: failed to add device: -17
[ 1039.129111] scsi 7:0:0:0: Direct-Access     LIO-ORG  IBLOCK           4.0  PQ: 0 ANSI: 5
[ 1039.129445] ------------[ cut here ]------------
[ 1039.129450] WARNING: CPU: 1 PID: 10604 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x64/0x80()
[ 1039.129451] sysfs: cannot create duplicate filename '/bus/scsi/devices/7:0:0:0'
[ 1039.129452] Modules linked in: target_core_pscsi target_core_file target_core_iblock xt_REDIRECT nf_nat_redirect xt_nat xt_mark xt_CHECKSUM nf_conntrack_ipv6 xt_mac xt_physdev xt_set xt_conntrack iptable_raw ip_set_hash_net ip_set nfnetlink veth ebtable_filter ebtables ip6table_filter ip6_tables iscsi_tcp libiscsi_tcp 8021q garp mrp rpcrdma ib_isert iscsi_target_mod bonding ib_iser libiscsi scsi_transport_iscsi openvswitch ib_srpt target_core_mod nf_defrag_ipv6 ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm xt_comment xt_multiport ib_sa iptable_filter ib_mad iptable_nat nf_conntrack_ipv4 usnic_verbs nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ib_core ib_addr iptable_mangle x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul
[ 1039.129483]  crc32_pclmul aesni_intel nfsd lrw gf128mul glue_helper ablk_helper ses cryptd enclosure ipmi_devintf sg sb_edac iTCO_wdt shpchp edac_core iTCO_vendor_support ipmi_si pcspkr lpc_ich ipmi_msghandler mfd_core auth_rpcgss acpi_power_meter wmi nfs_acl lockd br_netfilter grace sunrpc bridge stp llc dm_multipath ip_tables xfs libcrc32c sd_mod mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ixgbe(O) fnic i40e libfcoe igb crc32c_intel megaraid_sas enic libfc vxlan ip6_udp_tunnel udp_tunnel ptp scsi_transport_fc i2c_algo_bit i2c_core pps_core dca dm_mirror dm_region_hash dm_log dm_mod
[ 1039.129513] CPU: 1 PID: 10604 Comm: iscsiadm Tainted: G        W  O    4.4.0 #1
[ 1039.129514] Hardware name: Cisco Systems Inc UCSC-C240-M3S/UCSC-C240-M3S, BIOS C240M3.2.0.8.0.071620152208 07/16/2015
[ 1039.129515]  0000000000000000 00000000c52587c8 ffff885fa5c9b948 ffffffff81329090
[ 1039.129517]  ffff885fa5c9b990 ffff885fa5c9b980 ffffffff810862c6 ffff882e793b5000
[ 1039.129519]  ffff882f9fb97fa8 ffff885fa5329e10 0000000000000001 ffffffffffffffef
[ 1039.129530] Call Trace:
[ 1039.129535]  [<ffffffff81329090>] dump_stack+0x44/0x64
[ 1039.129538]  [<ffffffff810862c6>] warn_slowpath_common+0x86/0xc0
[ 1039.129539]  [<ffffffff8108635c>] warn_slowpath_fmt+0x5c/0x80
[ 1039.129542]  [<ffffffff8127e9a8>] ? kernfs_path+0x48/0x60
[ 1039.129544]  [<ffffffff81282014>] sysfs_warn_dup+0x64/0x80
[ 1039.129545]  [<ffffffff812823aa>] sysfs_do_create_link_sd.isra.2+0xaa/0xb0
[ 1039.129547]  [<ffffffff812823d5>] sysfs_create_link+0x25/0x40
[ 1039.129550]  [<ffffffff81454a6b>] bus_add_device+0x10b/0x1f0
[ 1039.129552]  [<ffffffff81452815>] device_add+0x3b5/0x610
[ 1039.129555]  [<ffffffff81487df5>] scsi_sysfs_add_sdev+0xa5/0x290
[ 1039.129557]  [<ffffffff81485355>] scsi_probe_and_add_lun+0xb65/0xd80
[ 1039.129561]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[ 1039.129562]  [<ffffffff81485d67>] __scsi_scan_target+0xf7/0x260
[ 1039.129564]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[ 1039.129566]  [<ffffffff81485fa7>] scsi_scan_target+0xd7/0xf0
[ 1039.129573]  [<ffffffffa07c29f5>] iscsi_user_scan_session.part.14+0x105/0x140 [scsi_transport_iscsi]
[ 1039.129576]  [<ffffffffa07c2a30>] ? iscsi_user_scan_session.part.14+0x140/0x140 [scsi_transport_iscsi]
[ 1039.129580]  [<ffffffffa07c2a4e>] iscsi_user_scan_session+0x1e/0x30 [scsi_transport_iscsi]
[ 1039.129581]  [<ffffffff81450b80>] device_for_each_child+0x50/0x90
[ 1039.129585]  [<ffffffffa07c083d>] iscsi_user_scan+0x3d/0x60 [scsi_transport_iscsi]
[ 1039.129587]  [<ffffffff81487bb6>] store_scan+0xa6/0x100
[ 1039.129591]  [<ffffffff811e3158>] ? __kmalloc+0x1b8/0x250
[ 1039.129594]  [<ffffffff81450318>] dev_attr_store+0x18/0x30
[ 1039.129596]  [<ffffffff8128170a>] sysfs_kf_write+0x3a/0x50
[ 1039.129597]  [<ffffffff81280d90>] kernfs_fop_write+0x120/0x170
[ 1039.129599]  [<ffffffff81203f17>] __vfs_write+0x37/0x100
[ 1039.129602]  [<ffffffff812b6c03>] ? selinux_file_permission+0xc3/0x110
[ 1039.129605]  [<ffffffff812ae75d>] ? security_file_permission+0x3d/0xc0
[ 1039.129607]  [<ffffffff810cd95f>] ? percpu_down_read+0x1f/0x50
[ 1039.129609]  [<ffffffff812045f2>] vfs_write+0xa2/0x1a0
[ 1039.129611]  [<ffffffff81003176>] ? do_audit_syscall_entry+0x66/0x70
[ 1039.129612]  [<ffffffff812053b5>] SyS_write+0x55/0xc0
[ 1039.129615]  [<ffffffff816945ae>] entry_SYSCALL_64_fastpath+0x12/0x71
[ 1039.129616] ---[ end trace bcfb1475313b5f65 ]---
[ 1039.129638] scsi 7:0:0:0: failed to add device: -17
[ 1048.204151] scsi 7:0:0:0: Direct-Access     LIO-ORG  IBLOCK           4.0  PQ: 0 ANSI: 5
[ 1048.204524] ------------[ cut here ]------------
[ 1048.204530] WARNING: CPU: 2 PID: 10624 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x64/0x80()
[ 1048.204531] sysfs: cannot create duplicate filename '/bus/scsi/devices/7:0:0:0'
[ 1048.204532] Modules linked in: target_core_pscsi target_core_file target_core_iblock xt_REDIRECT nf_nat_redirect xt_nat xt_mark xt_CHECKSUM nf_conntrack_ipv6 xt_mac xt_physdev xt_set xt_conntrack iptable_raw ip_set_hash_net ip_set nfnetlink veth ebtable_filter ebtables ip6table_filter ip6_tables iscsi_tcp libiscsi_tcp 8021q garp mrp rpcrdma ib_isert iscsi_target_mod bonding ib_iser libiscsi scsi_transport_iscsi openvswitch ib_srpt target_core_mod nf_defrag_ipv6 ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm xt_comment xt_multiport ib_sa iptable_filter ib_mad iptable_nat nf_conntrack_ipv4 usnic_verbs nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ib_core ib_addr iptable_mangle x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul
[ 1048.204564]  crc32_pclmul aesni_intel nfsd lrw gf128mul glue_helper ablk_helper ses cryptd enclosure ipmi_devintf sg sb_edac iTCO_wdt shpchp edac_core iTCO_vendor_support ipmi_si pcspkr lpc_ich ipmi_msghandler mfd_core auth_rpcgss acpi_power_meter wmi nfs_acl lockd br_netfilter grace sunrpc bridge stp llc dm_multipath ip_tables xfs libcrc32c sd_mod mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ixgbe(O) fnic i40e libfcoe igb crc32c_intel megaraid_sas enic libfc vxlan ip6_udp_tunnel udp_tunnel ptp scsi_transport_fc i2c_algo_bit i2c_core pps_core dca dm_mirror dm_region_hash dm_log dm_mod
[ 1048.204597] CPU: 2 PID: 10624 Comm: iscsiadm Tainted: G        W  O    4.4.0 #1
[ 1048.204599] Hardware name: Cisco Systems Inc UCSC-C240-M3S/UCSC-C240-M3S, BIOS C240M3.2.0.8.0.071620152208 07/16/2015
[ 1048.204600]  0000000000000000 0000000088d9f919 ffff882e7aa0b948 ffffffff81329090
[ 1048.204602]  ffff882e7aa0b990 ffff882e7aa0b980 ffffffff810862c6 ffff882e7b4f1000
[ 1048.204604]  ffff885f9f16f2c8 ffff885fa5329e10 0000000000000001 ffffffffffffffef
[ 1048.204605] Call Trace:
[ 1048.204612]  [<ffffffff81329090>] dump_stack+0x44/0x64
[ 1048.204614]  [<ffffffff810862c6>] warn_slowpath_common+0x86/0xc0
[ 1048.204616]  [<ffffffff8108635c>] warn_slowpath_fmt+0x5c/0x80
[ 1048.204619]  [<ffffffff8127e9a8>] ? kernfs_path+0x48/0x60
[ 1048.204620]  [<ffffffff81282014>] sysfs_warn_dup+0x64/0x80
[ 1048.204622]  [<ffffffff812823aa>] sysfs_do_create_link_sd.isra.2+0xaa/0xb0
[ 1048.204624]  [<ffffffff812823d5>] sysfs_create_link+0x25/0x40
[ 1048.204627]  [<ffffffff81454a6b>] bus_add_device+0x10b/0x1f0
[ 1048.204628]  [<ffffffff81452815>] device_add+0x3b5/0x610
[ 1048.204632]  [<ffffffff81487df5>] scsi_sysfs_add_sdev+0xa5/0x290
[ 1048.204634]  [<ffffffff81485355>] scsi_probe_and_add_lun+0xb65/0xd80
[ 1048.204638]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[ 1048.204640]  [<ffffffff81485d67>] __scsi_scan_target+0xf7/0x260
[ 1048.204641]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[ 1048.204643]  [<ffffffff81485fa7>] scsi_scan_target+0xd7/0xf0
[ 1048.204651]  [<ffffffffa07c29f5>] iscsi_user_scan_session.part.14+0x105/0x140 [scsi_transport_iscsi]
[ 1048.204654]  [<ffffffffa07c2a30>] ? iscsi_user_scan_session.part.14+0x140/0x140 [scsi_transport_iscsi]
[ 1048.204658]  [<ffffffffa07c2a4e>] iscsi_user_scan_session+0x1e/0x30 [scsi_transport_iscsi]
[ 1048.204659]  [<ffffffff81450b80>] device_for_each_child+0x50/0x90
[ 1048.204663]  [<ffffffffa07c083d>] iscsi_user_scan+0x3d/0x60 [scsi_transport_iscsi]
[ 1048.204665]  [<ffffffff81487bb6>] store_scan+0xa6/0x100
[ 1048.204669]  [<ffffffff811e3158>] ? __kmalloc+0x1b8/0x250
[ 1048.204673]  [<ffffffff81450318>] dev_attr_store+0x18/0x30
[ 1048.204674]  [<ffffffff8128170a>] sysfs_kf_write+0x3a/0x50
[ 1048.204676]  [<ffffffff81280d90>] kernfs_fop_write+0x120/0x170
[ 1048.204678]  [<ffffffff81203f17>] __vfs_write+0x37/0x100
[ 1048.204680]  [<ffffffff812b6c03>] ? selinux_file_permission+0xc3/0x110
[ 1048.204684]  [<ffffffff812ae75d>] ? security_file_permission+0x3d/0xc0
[ 1048.204687]  [<ffffffff810cd95f>] ? percpu_down_read+0x1f/0x50
[ 1048.204688]  [<ffffffff812045f2>] vfs_write+0xa2/0x1a0
[ 1048.204690]  [<ffffffff81003176>] ? do_audit_syscall_entry+0x66/0x70
[ 1048.204692]  [<ffffffff812053b5>] SyS_write+0x55/0xc0
[ 1048.204695]  [<ffffffff816945ae>] entry_SYSCALL_64_fastpath+0x12/0x71
[ 1048.204697] ---[ end trace bcfb1475313b5f66 ]---
[ 1048.204719] scsi 7:0:0:0: failed to add device: -17
[ 1064.276300] scsi 7:0:0:0: Direct-Access     LIO-ORG  IBLOCK           4.0  PQ: 0 ANSI: 5
[ 1064.276598] ------------[ cut here ]------------
[ 1064.276603] WARNING: CPU: 1 PID: 10652 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x64/0x80()
[ 1064.276604] sysfs: cannot create duplicate filename '/bus/scsi/devices/7:0:0:0'
[ 1064.276604] Modules linked in: target_core_pscsi target_core_file target_core_iblock xt_REDIRECT nf_nat_redirect xt_nat xt_mark xt_CHECKSUM nf_conntrack_ipv6 xt_mac xt_physdev xt_set xt_conntrack iptable_raw ip_set_hash_net ip_set nfnetlink veth ebtable_filter ebtables ip6table_filter ip6_tables iscsi_tcp libiscsi_tcp 8021q garp mrp rpcrdma ib_isert iscsi_target_mod bonding ib_iser libiscsi scsi_transport_iscsi openvswitch ib_srpt target_core_mod nf_defrag_ipv6 ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm xt_comment xt_multiport ib_sa iptable_filter ib_mad iptable_nat nf_conntrack_ipv4 usnic_verbs nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ib_core ib_addr iptable_mangle x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul
[ 1064.276649]  crc32_pclmul aesni_intel nfsd lrw gf128mul glue_helper ablk_helper ses cryptd enclosure ipmi_devintf sg sb_edac iTCO_wdt shpchp edac_core iTCO_vendor_support ipmi_si pcspkr lpc_ich ipmi_msghandler mfd_core auth_rpcgss acpi_power_meter wmi nfs_acl lockd br_netfilter grace sunrpc bridge stp llc dm_multipath ip_tables xfs libcrc32c sd_mod mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ixgbe(O) fnic i40e libfcoe igb crc32c_intel megaraid_sas enic libfc vxlan ip6_udp_tunnel udp_tunnel ptp scsi_transport_fc i2c_algo_bit i2c_core pps_core dca dm_mirror dm_region_hash dm_log dm_mod
[ 1064.276682] CPU: 1 PID: 10652 Comm: iscsiadm Tainted: G        W  O    4.4.0 #1
[ 1064.276683] Hardware name: Cisco Systems Inc UCSC-C240-M3S/UCSC-C240-M3S, BIOS C240M3.2.0.8.0.071620152208 07/16/2015
[ 1064.276684]  0000000000000000 0000000080e19618 ffff882e7aa0b948 ffffffff81329090
[ 1064.276686]  ffff882e7aa0b990 ffff882e7aa0b980 ffffffff810862c6 ffff882f9fed7000
[ 1064.276688]  ffff885f9f16f2d0 ffff885fa5329e10 0000000000000001 ffffffffffffffef
[ 1064.276689] Call Trace:
[ 1064.276695]  [<ffffffff81329090>] dump_stack+0x44/0x64
[ 1064.276698]  [<ffffffff810862c6>] warn_slowpath_common+0x86/0xc0
[ 1064.276700]  [<ffffffff8108635c>] warn_slowpath_fmt+0x5c/0x80
[ 1064.276703]  [<ffffffff8127e9a8>] ? kernfs_path+0x48/0x60
[ 1064.276704]  [<ffffffff81282014>] sysfs_warn_dup+0x64/0x80
[ 1064.276706]  [<ffffffff812823aa>] sysfs_do_create_link_sd.isra.2+0xaa/0xb0
[ 1064.276708]  [<ffffffff812823d5>] sysfs_create_link+0x25/0x40
[ 1064.276711]  [<ffffffff81454a6b>] bus_add_device+0x10b/0x1f0
[ 1064.276712]  [<ffffffff81452815>] device_add+0x3b5/0x610
[ 1064.276716]  [<ffffffff81487df5>] scsi_sysfs_add_sdev+0xa5/0x290
[ 1064.276718]  [<ffffffff81485355>] scsi_probe_and_add_lun+0xb65/0xd80
[ 1064.276722]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[ 1064.276724]  [<ffffffff81485d67>] __scsi_scan_target+0xf7/0x260
[ 1064.276725]  [<ffffffff814608ac>] ? __pm_runtime_resume+0x5c/0x70
[ 1064.276727]  [<ffffffff81485fa7>] scsi_scan_target+0xd7/0xf0
[ 1064.276734]  [<ffffffffa07c29f5>] iscsi_user_scan_session.part.14+0x105/0x140 [scsi_transport_iscsi]
[ 1064.276738]  [<ffffffffa07c2a30>] ? iscsi_user_scan_session.part.14+0x140/0x140 [scsi_transport_iscsi]
[ 1064.276742]  [<ffffffffa07c2a4e>] iscsi_user_scan_session+0x1e/0x30 [scsi_transport_iscsi]
[ 1064.276743]  [<ffffffff81450b80>] device_for_each_child+0x50/0x90
[ 1064.276747]  [<ffffffffa07c083d>] iscsi_user_scan+0x3d/0x60 [scsi_transport_iscsi]
[ 1064.276749]  [<ffffffff81487bb6>] store_scan+0xa6/0x100
[ 1064.276753]  [<ffffffff811e3158>] ? __kmalloc+0x1b8/0x250
[ 1064.276757]  [<ffffffff81450318>] dev_attr_store+0x18/0x30
[ 1064.276759]  [<ffffffff8128170a>] sysfs_kf_write+0x3a/0x50
[ 1064.276760]  [<ffffffff81280d90>] kernfs_fop_write+0x120/0x170
[ 1064.276762]  [<ffffffff81203f17>] __vfs_write+0x37/0x100
[ 1064.276765]  [<ffffffff812b6c03>] ? selinux_file_permission+0xc3/0x110
[ 1064.276769]  [<ffffffff812ae75d>] ? security_file_permission+0x3d/0xc0
[ 1064.276772]  [<ffffffff810cd95f>] ? percpu_down_read+0x1f/0x50
[ 1064.276773]  [<ffffffff812045f2>] vfs_write+0xa2/0x1a0
[ 1064.276776]  [<ffffffff81003176>] ? do_audit_syscall_entry+0x66/0x70
[ 1064.276777]  [<ffffffff812053b5>] SyS_write+0x55/0xc0
[ 1064.276780]  [<ffffffff816945ae>] entry_SYSCALL_64_fastpath+0x12/0x71
[ 1064.276781] ---[ end trace bcfb1475313b5f67 ]---
[ 1064.276817] scsi 7:0:0:0: failed to add device: -17


                                                               

Thank you

Serguei


-----Original Message-----
From: Michael Christie [mailto:michaelc@cs.wisc.edu] 
Sent: Friday, January 29, 2016 3:02 AM
To: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>
Cc: bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org
Subject: Re: [Bug 111441] New: iscsi fails to attach to targets

Can you send me your lio config?

Are you running lio on the same box as the initiator so it is also changing kernel version with each test?

It looks like the scsi scan code is not handling the lack of LUN 0 correctly. I think I know what patch is causing it, but cannot get my lio config to return the same info as you for the non existent LUNs, so I think I am hitting slightly different error paths.


> On Jan 28, 2016, at 8:55 PM, Serguei Bezverkhi (sbezverk)
> <sbezverk@cisco.com> wrote:
> 
> HI Mike,
> 
> Thank you for looking into this issue.
> I reproduced this issue  this using both mainline compiled kernel and the one
> posted by El Repo.  I used both of these kernels with RHEL 7.2, both kernels
> showed exactly the same issue. When I rollback to the original kernel
> 3.10.0-327.4.4, I do not see the issue.
> 
> As per your request attaching kernel config file. Please let me know if you
> need any additional info or if you want to take a look at the router, I can
> setup a webex meeting to show you the issue.
> 
> Thank you
> 
> Serguei
> 
> 
> 
> 
> Serguei Bezverkhi,
> TECHNICAL LEADER.SERVICES
> Global SP Services
> sbezverk@cisco.com
> Phone: +1 416 306 7312
> Mobile: +1 514 234 7374
> 
> CCIE (R&S,SP,Sec) - #9527
> 
> Cisco.com
> 
> 
> 
>  Think before you print.
> This email may contain confidential and privileged material for the sole use
> of the intended recipient. Any review, use, distribution or disclosure by
> others is strictly prohibited. If you are not the intended recipient (or
> authorized to receive for the recipient), please contact the sender by reply
> email and delete all copies of this message.
> Please click here for Company Registration Information.
> 
> 
> 
> 
> -----Original Message-----
> From: linux-scsi-owner@vger.kernel.org
> [mailto:linux-scsi-owner@vger.kernel.org] On Behalf Of Mike Christie
> Sent: Thursday, January 28, 2016 8:53 PM
> To: bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org
> Subject: Re: [Bug 111441] New: iscsi fails to attach to targets
> 
> On 01/28/2016 04:51 PM, bugzilla-daemon@bugzilla.kernel.org wrote:
>> https://bugzilla.kernel.org/show_bug.cgi?id=111441
>> 
>>            Bug ID: 111441
>>           Summary: iscsi fails to attach to targets
>>           Product: IO/Storage
>>           Version: 2.5
>>    Kernel Version: 4.4.0-1
>>          Hardware: x86-64
>>                OS: Linux
>>              Tree: Mainline
>>            Status: NEW
> 
> 
>> 4.4.0-1.el7.elrepo.x86_64 #1
> 
> I have not seen this oops before. We saw similar ones around 5 or 6 years
> ago, but they were due to some sysfs or block or scsi issue (I cannot
> remember exacty) and fixed there.
> 
> Is this a distro kernel or mainline. BZ says mainline, but the kernel name in
> bug looks like a red hat related one. Where did you get it?
> 
> I do not hit this in 4.4 mainline. Send me your kernel .config, so I can make
> sure I have the same options.
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the
> body of a message to majordomo@vger.kernel.org More majordomo info at 
> http://vger.kernel.org/majordomo-info.html

If you send me the diff for your patch, I will build new kernel myself.

Serguei


Serguei Bezverkhi,
TECHNICAL LEADER.SERVICES
Global SP Services
sbezverk@cisco.com
Phone: +1 416 306 7312
Mobile: +1 514 234 7374

CCIE (R&S,SP,Sec) - #9527

Cisco.com



 Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here for Company Registration Information.




-----Original Message-----
From: Michael Christie [mailto:michaelc@cs.wisc.edu] 
Sent: Friday, January 29, 2016 2:09 PM
To: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>
Cc: bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org
Subject: Re: [Bug 111441] New: iscsi fails to attach to targets


> On Jan 29, 2016, at 6:04 AM, Serguei Bezverkhi (sbezverk)
> <sbezverk@cisco.com> wrote:
> 
> Actually this server uses both cases: Local taregts (since it is OpenStack
> server) and remote targets as it tries to mount 4 remotefile systems.  
> 
> You are correct, I always use the same box I just change the kernel it is
> using to boot. No other changes to the environment. I do not mind to load a
> test kernel without that suspected patch, just get me the RPM.
> 

I do not know what you mean. I think the patch I sent will fix the sysfs errors caused due to alua not being setup properly on your system and scsi_dh_alua failing to attach. That patch should be applied to the 4.4 upstream kernel. Are you saying you want me to make you a kernel rpm?

HI Mike,

I tried your patch and it is has eliminated first traceback but I still do not see my remote targets.


Here is dmesg

[   26.103812] scsi 3:0:0:2: Direct-Access     LIO-ORG  san-disk-2       4.0  PQ: 0 ANSI: 5
[   26.104338] sd 3:0:0:2: alua: supports implicit and explicit TPGS
[   26.104549] sd 3:0:0:2: alua: No target port descriptors found
[   26.104552] sd 3:0:0:2: alua: Attach failed (-22)
[   26.104554] sd 3:0:0:2: failed to add device handler: -22
[   26.104578] sd 3:0:0:2: [sdc] 20507809792 512-byte logical blocks: (10.4 TB/9.54 TiB)
[   26.104905] sd 3:0:0:2: [sdc] Write Protect is off
[   26.104908] sd 3:0:0:2: [sdc] Mode Sense: 43 00 10 08
[   26.105036] sd 3:0:0:2: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA
[   26.112294] scsi host6: iSCSI Initiator over TCP/IP
[   26.113279] scsi 4:0:0:3: Direct-Access     LIO-ORG  san-disk-3       4.0  PQ: 0 ANSI: 5
[   26.113690] sd 4:0:0:3: alua: supports implicit and explicit TPGS
[   26.113877] sd 4:0:0:3: [sdd] 9765625856 512-byte logical blocks: (5.00 TB/4.54 TiB)
[   26.113948] sd 4:0:0:3: alua: No target port descriptors found
[   26.113951] sd 4:0:0:3: alua: Attach failed (-22)
[   26.113953] sd 4:0:0:3: failed to add device handler: -22
[   26.114292] sd 4:0:0:3: [sdd] Write Protect is off
[   26.114295] sd 4:0:0:3: [sdd] Mode Sense: 43 00 10 08
[   26.114503] sd 4:0:0:3: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA
[   26.123875] scsi 5:0:0:1: Direct-Access     LIO-ORG  san-disk-1       4.0  PQ: 0 ANSI: 5
[   26.123911] scsi 6:0:0:4: Direct-Access     LIO-ORG  san-disk-4       4.0  PQ: 0 ANSI: 5
[   26.124452] sd 6:0:0:4: alua: supports implicit and explicit TPGS
[   26.124453] sd 5:0:0:1: alua: supports implicit and explicit TPGS
[   26.124724] sd 5:0:0:1: alua: No target port descriptors found
[   26.124727] sd 5:0:0:1: alua: Attach failed (-22)
[   26.124728] sd 5:0:0:1: failed to add device handler: -22
[   26.124736] sd 6:0:0:4: [sde] 10742171648 512-byte logical blocks: (5.49 TB/5.00 TiB)
[   26.124773] sd 5:0:0:1: [sdf] 7812499389 512-byte logical blocks: (3.99 TB/3.63 TiB)
[   26.124777] sd 6:0:0:4: alua: No target port descriptors found
[   26.124779] sd 6:0:0:4: alua: Attach failed (-22)
[   26.124780] sd 6:0:0:4: failed to add device handler: -22
[   26.125182] sd 5:0:0:1: [sdf] Write Protect is off
[   26.125184] sd 5:0:0:1: [sdf] Mode Sense: 43 00 10 08
[   26.125217] sd 6:0:0:4: [sde] Write Protect is off
[   26.125220] sd 6:0:0:4: [sde] Mode Sense: 43 00 10 08
[   26.125306] sd 5:0:0:1: [sdf] Write cache: enabled, read cache: enabled, supports DPO and FUA
[   26.125512] sd 6:0:0:4: [sde] Write cache: enabled, read cache: enabled, supports DPO and FUA
[   26.129633]  sdf: sdf1
[   26.130637] sd 5:0:0:1: [sdf] Attached SCSI disk
[   26.144377] ixgbe 0000:04:00.0: registered PHC device on enp4s0f0
[   26.149072]  sdc: sdc1
[   26.150434] sd 3:0:0:2: [sdc] Attached SCSI disk
[   26.190709]  sdd: sdd1 sdd2
[   26.193348] sd 4:0:0:3: [sdd] Attached SCSI disk
[   26.230515]  sde: sde1
[   26.231674] sd 6:0:0:4: [sde] Attached SCSI disk
[   26.231987] sd 6:0:0:4: [sde] Synchronizing SCSI cache
[   26.232021] sd 5:0:0:1: [sdf] Synchronizing SCSI cache
[   26.233212] sd 3:0:0:2: [sdc] Synchronizing SCSI cache
[   26.233440] sd 4:0:0:3: [sdd] Synchronizing SCSI cache
[   26.236755] Buffer I/O error on dev sdc, logical block 2563476132, async page read
[   26.238897] Buffer I/O error on dev sdd, logical block 1220703182, async page read
[   26.245773] ixgbe 0000:04:00.1: SR-IOV enabled with 8 VFs
[   26.245775] ixgbe 0000:04:00.1: configure port vlans to keep your VFs secure
[   26.274544] scsi 6:0:0:0: Unexpected response from lun 4 while scanning, scan aborted
[   26.283173] scsi 3:0:0:0: Unexpected response from lun 2 while scanning, scan aborted
[   26.288571] scsi 4:0:0:0: Unexpected response from lun 3 while scanning, scan aborted
[   26.288618] scsi 5:0:0:0: Unexpected response from lun 1 while scanning, scan aborted


Second traceback is gone too, but still no luck attaching local iscsi targets either.


[  639.148875] TARGET_CORE[iSCSI]: Expected Transfer Length: 264 does not match SCSI CDB Length: 8 for SAM Opcode: 0x12
[  639.148911] sd 7:0:0:0: [sdc] 115343360 512-byte logical blocks: (59.0 GB/55.0 GiB)
[  639.148925] sd 7:0:0:0: alua: No target port descriptors found
[  639.148928] sd 7:0:0:0: alua: Attach failed (-22)
[  639.149186] sd 7:0:0:0: [sdc] Write Protect is off
[  639.149188] sd 7:0:0:0: [sdc] Mode Sense: 43 00 10 08
[  639.149279] sd 7:0:0:0: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA
[  639.149298] iSCSI/iqn.1994-05.com.redhat:cf7f1fafca4b: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[  639.149530] sd 7:0:0:0: failed to add device handler: -22
[  639.154762] sd 7:0:0:0: [sdc] Attached SCSI disk
[  639.154857] sd 7:0:0:0: [sdc] Synchronizing SCSI cache
[  655.279047] scsi 7:0:0:0: Direct-Access     LIO-ORG  IBLOCK           4.0  PQ: 0 ANSI: 5
[  655.279397] sd 7:0:0:0: alua: supports implicit and explicit TPGS
[  655.279503] TARGET_CORE[iSCSI]: Expected Transfer Length: 264 does not match SCSI CDB Length: 8 for SAM Opcode: 0x12
[  655.279533] sd 7:0:0:0: alua: No target port descriptors found
[  655.279535] sd 7:0:0:0: alua: Attach failed (-22)
[  655.279587] sd 7:0:0:0: [sdc] 115343360 512-byte logical blocks: (59.0 GB/55.0 GiB)
[  655.279848] sd 7:0:0:0: [sdc] Write Protect is off
[  655.279849] sd 7:0:0:0: [sdc] Mode Sense: 43 00 10 08
[  655.279981] sd 7:0:0:0: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA
[  655.280034] iSCSI/iqn.1994-05.com.redhat:cf7f1fafca4b: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[  655.280171] sd 7:0:0:0: failed to add device handler: -22
[  655.286008] sd 7:0:0:0: [sdc] Attached SCSI disk
[  655.286132] sd 7:0:0:0: [sdc] Synchronizing SCSI cache


Serguei Bezverkhi,
TECHNICAL LEADER.SERVICES
Global SP Services
sbezverk@cisco.com
Phone: +1 416 306 7312
Mobile: +1 514 234 7374

CCIE (R&S,SP,Sec) - #9527

Cisco.com



 Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here for Company Registration Information.




-----Original Message-----
From: Mike Christie [mailto:michaelc@cs.wisc.edu] 
Sent: Friday, January 29, 2016 2:27 PM
To: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>
Cc: bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org
Subject: Re: [Bug 111441] New: iscsi fails to attach to targets



On 01/29/2016 01:11 PM, Serguei Bezverkhi (sbezverk) wrote:
> If you send me the diff for your patch, I will build new kernel myself.
> 

Bugzilla must be messing something up. I attached to one of the previous mails. Attaching it here again.

Email me offlist and without bugzilla if you do not get it here.

The patch will fix the syfs bug ons you are hitting.

I am not sure if it will fix the genhd one. We can deal with that one next if it is a different issue.


> Serguei
> 
> 
> Serguei Bezverkhi,
> TECHNICAL LEADER.SERVICES
> Global SP Services
> sbezverk@cisco.com
> Phone: +1 416 306 7312
> Mobile: +1 514 234 7374
> 
> CCIE (R&S,SP,Sec) - #9527
> 
> Cisco.com
> 
> 
> 
>  Think before you print.
> This email may contain confidential and privileged material for the sole use
> of the intended recipient. Any review, use, distribution or disclosure by
> others is strictly prohibited. If you are not the intended recipient (or
> authorized to receive for the recipient), please contact the sender by reply
> email and delete all copies of this message.
> Please click here for Company Registration Information.
> 
> 
> 
> 
> -----Original Message-----
> From: Michael Christie [mailto:michaelc@cs.wisc.edu]
> Sent: Friday, January 29, 2016 2:09 PM
> To: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>
> Cc: bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org
> Subject: Re: [Bug 111441] New: iscsi fails to attach to targets
> 
> 
>> On Jan 29, 2016, at 6:04 AM, Serguei Bezverkhi (sbezverk)
>> <sbezverk@cisco.com> wrote:
>>
>> Actually this server uses both cases: Local taregts (since it is OpenStack
>> server) and remote targets as it tries to mount 4 remotefile systems.  
>>
>> You are correct, I always use the same box I just change the kernel it is
>> using to boot. No other changes to the environment. I do not mind to load a
>> test kernel without that suspected patch, just get me the RPM.
>>
> 
> I do not know what you mean. I think the patch I sent will fix the sysfs
> errors caused due to alua not being setup properly on your system and
> scsi_dh_alua failing to attach. That patch should be applied to the 4.4
> upstream kernel. Are you saying you want me to make you a kernel rpm?
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" 
> in the body of a message to majordomo@vger.kernel.org More majordomo 
> info at  http://vger.kernel.org/majordomo-info.html
>

On Fri, 2016-01-29 at 17:32 -0600, Mike Christie wrote:
> On 01/29/2016 04:21 PM, Serguei Bezverkhi (sbezverk) wrote:
> > HI Mike,
> > 
> > I tried your patch and it is has eliminated first traceback but I still do
> not see my remote targets.
> > 
> 
> That is sort of expected. Your target is not setup for ALUA properly. It
> says it supports ALUA, but when scsi_dh_alua asks about the ports it is
> reporting there are none. Ccing the people that made the patch that
> added the issue and own the code.
> 
> Hey Christoph and Hannes,
> 
> The dh/alua changes that added this:
> 
>         error = scsi_dh_add_device(sdev);
>         if (error) {
>                 sdev_printk(KERN_INFO, sdev,
>                                 "failed to add device handler: %d\n",
> error);
>                 return error;
>         }
> 
> to scsi_sysfs_add_sdev are adding a regression.
> 
> 1. If that fails, then we forget to do device_del before doing the
> return. My patch in this thread added that back, so we do not see the
> sysfs oopses anymore. But.....
> 
> 2. It looks like in older kernels, we would allow misconfigured targets
> like this one to still setup devices. Do we want that old behavior back?
> Should we just ignore the return value from scsi_dh_add_device above?
> Note that in this case, it is LIO so it can be easily fixed on the
> target side by just setting it up properly. I do not think other targets
> would hit this type of issue.
> 

Btw, what does misconfigured mean here wrt target ALUA..?

Created attachment 202531 [details]
sbezverk-osp3-debug.log

Hi Mike,

I have built a new kernel with debug and collected dmesg, hopefully this log has more info to understand why alua fails to attach remote targets with error -22.

Please see attached log.

Thank you

Serguei


Serguei Bezverkhi,
TECHNICAL LEADER.SERVICES
Global SP Services
sbezverk@cisco.com
Phone: +1 416 306 7312
Mobile: +1 514 234 7374

CCIE (R&S,SP,Sec) - #9527

Cisco.com



 Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here for Company Registration Information.




-----Original Message-----
From: Nicholas A. Bellinger [mailto:nab@linux-iscsi.org] 
Sent: Saturday, January 30, 2016 2:39 AM
To: Mike Christie <michaelc@cs.wisc.edu>
Cc: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>; bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org; Christoph Hellwig <hch@infradead.org>; Hannes Reinecke <hare@suse.de>
Subject: Re: [Bug 111441] New: iscsi fails to attach to targets

On Fri, 2016-01-29 at 17:32 -0600, Mike Christie wrote:
> On 01/29/2016 04:21 PM, Serguei Bezverkhi (sbezverk) wrote:
> > HI Mike,
> > 
> > I tried your patch and it is has eliminated first traceback but I still do
> not see my remote targets.
> > 
> 
> That is sort of expected. Your target is not setup for ALUA properly. 
> It says it supports ALUA, but when scsi_dh_alua asks about the ports 
> it is reporting there are none. Ccing the people that made the patch 
> that added the issue and own the code.
> 
> Hey Christoph and Hannes,
> 
> The dh/alua changes that added this:
> 
>         error = scsi_dh_add_device(sdev);
>         if (error) {
>                 sdev_printk(KERN_INFO, sdev,
>                                 "failed to add device handler: %d\n", 
> error);
>                 return error;
>         }
> 
> to scsi_sysfs_add_sdev are adding a regression.
> 
> 1. If that fails, then we forget to do device_del before doing the 
> return. My patch in this thread added that back, so we do not see the 
> sysfs oopses anymore. But.....
> 
> 2. It looks like in older kernels, we would allow misconfigured 
> targets like this one to still setup devices. Do we want that old behavior
> back?
> Should we just ignore the return value from scsi_dh_add_device above?
> Note that in this case, it is LIO so it can be easily fixed on the 
> target side by just setting it up properly. I do not think other 
> targets would hit this type of issue.
> 

Btw, what does misconfigured mean here wrt target ALUA..?

Hello,

Any chance we could move forward with this investigation? I still cannot attach to any remove iscsi targets with either 4.4.0 or 4.4.1 kernels.

Thank you

Serguei


 

-----Original Message-----
From: Mike Christie [mailto:michaelc@cs.wisc.edu] 
Sent: Monday, February 01, 2016 11:55 AM
To: Nicholas A. Bellinger <nab@linux-iscsi.org>
Cc: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>; bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org; Christoph Hellwig <hch@infradead.org>; Hannes Reinecke <hare@suse.de>
Subject: Re: [Bug 111441] New: iscsi fails to attach to targets

On 01/30/2016 01:38 AM, Nicholas A. Bellinger wrote:
> On Fri, 2016-01-29 at 17:32 -0600, Mike Christie wrote:
>> On 01/29/2016 04:21 PM, Serguei Bezverkhi (sbezverk) wrote:
>>> HI Mike,
>>>
>>> I tried your patch and it is has eliminated first traceback but I still do
>>> not see my remote targets.
>>>
>>
>> That is sort of expected. Your target is not setup for ALUA properly. 
>> It says it supports ALUA, but when scsi_dh_alua asks about the ports 
>> it is reporting there are none. Ccing the people that made the patch 
>> that added the issue and own the code.
>>
>> Hey Christoph and Hannes,
>>
>> The dh/alua changes that added this:
>>
>>         error = scsi_dh_add_device(sdev);
>>         if (error) {
>>                 sdev_printk(KERN_INFO, sdev,
>>                                 "failed to add device handler: %d\n", 
>> error);
>>                 return error;
>>         }
>>
>> to scsi_sysfs_add_sdev are adding a regression.
>>
>> 1. If that fails, then we forget to do device_del before doing the 
>> return. My patch in this thread added that back, so we do not see the 
>> sysfs oopses anymore. But.....
>>
>> 2. It looks like in older kernels, we would allow misconfigured 
>> targets like this one to still setup devices. Do we want that old behavior
>> back?
>> Should we just ignore the return value from scsi_dh_add_device above?
>> Note that in this case, it is LIO so it can be easily fixed on the 
>> target side by just setting it up properly. I do not think other 
>> targets would hit this type of issue.
>>
> 
> Btw, what does misconfigured mean here wrt target ALUA..?

[   25.833195] sd 6:0:0:4: alua: supports implicit and explicit TPGS
[   25.833360] sd 6:0:0:4: alua: No target port descriptors found
[   25.833363] sd 6:0:0:4: alua: Attach failed (-22)
[   25.833365] sd 6:0:0:4: failed to add device handler: -22

He has LIO configured to report it supports implicit/explicit ALUA, but the ports do not seem to be configured.

For the LIO config side, are his LUNs just not in a the default_lu_gp or any other group?

Sure thing, I will test it tonight and let you know the result.

Thank you

Serguei

-----Original Message-----
From: Mike Christie [mailto:michaelc@cs.wisc.edu] 
Sent: Tuesday, February 02, 2016 3:12 PM
To: Christoph Hellwig <hch@infradead.org>
Cc: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>; bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org; Hannes Reinecke <hare@suse.de>
Subject: Re: [Bug 111441] New: iscsi fails to attach to targets

On 02/02/2016 12:09 PM, Christoph Hellwig wrote:
> On Fri, Jan 29, 2016 at 05:32:54PM -0600, Mike Christie wrote:
>> Hey Christoph and Hannes,
>>
>> The dh/alua changes that added this:
>>
>>         error = scsi_dh_add_device(sdev);
>>         if (error) {
>>                 sdev_printk(KERN_INFO, sdev,
>>                                 "failed to add device handler: %d\n", 
>> error);
>>                 return error;
>>         }
>>
>> to scsi_sysfs_add_sdev are adding a regression.
>>
>> 1. If that fails, then we forget to do device_del before doing the 
>> return. My patch in this thread added that back, so we do not see the 
>> sysfs oopses anymore. But.....
> 
> Ok.
> 
>> 2. It looks like in older kernels, we would allow misconfigured 
>> targets like this one to still setup devices. Do we want that old behavior
>> back?
>> Should we just ignore the return value from scsi_dh_add_device above?
>> Note that in this case, it is LIO so it can be easily fixed on the 
>> target side by just setting it up properly. I do not think other 
>> targets would hit this type of issue.
> 
> Be liberal in what you accept..  I guess we need to continue allowing 
> to connect to these broken targets, but a warning would be useful.
> 
> Can you send a patch?

Serguei, can you try the attached patch? Drop the other one I sent.

HI Mike,

It is working now, it spits alua related error messages but it attaches remote and local targets, so my OpenStack is back to life. Please let me know if you plan to commit this fix upstream.

[   27.579303] scsi 3:0:0:2: Direct-Access     LIO-ORG  san-disk-2       4.0  PQ: 0 ANSI: 5
[   27.579797] sd 3:0:0:2: alua: supports implicit and explicit TPGS
[   27.579932] sd 3:0:0:2: [sdc] 20507809792 512-byte logical blocks: (10.4 TB/9.54 TiB)
[   27.579975] sd 3:0:0:2: alua: No target port descriptors found
[   27.579977] sd 3:0:0:2: alua: Attach failed (-22)
[   27.580640] sd 3:0:0:2: failed to add device handler: -22
[   27.580897] sd 3:0:0:2: [sdc] Write Protect is off
[   27.580899] sd 3:0:0:2: [sdc] Mode Sense: 43 00 10 08
[   27.580922] sd 3:0:0:2: Attached scsi generic sg3 type 0
[   27.581041] sd 3:0:0:2: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA
[   27.630156] sd 3:0:0:2: [sdc] Attached SCSI disk

Thank you very much

Serguei


Serguei Bezverkhi,
TECHNICAL LEADER.SERVICES
Global SP Services
sbezverk@cisco.com
Phone: +1 416 306 7312
Mobile: +1 514 234 7374

CCIE (R&S,SP,Sec) - #9527

Cisco.com



 Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here for Company Registration Information.




-----Original Message-----
From: Mike Christie [mailto:michaelc@cs.wisc.edu] 
Sent: Tuesday, February 02, 2016 3:12 PM
To: Christoph Hellwig <hch@infradead.org>
Cc: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>; bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org; Hannes Reinecke <hare@suse.de>
Subject: Re: [Bug 111441] New: iscsi fails to attach to targets

On 02/02/2016 12:09 PM, Christoph Hellwig wrote:
> On Fri, Jan 29, 2016 at 05:32:54PM -0600, Mike Christie wrote:
>> Hey Christoph and Hannes,
>>
>> The dh/alua changes that added this:
>>
>>         error = scsi_dh_add_device(sdev);
>>         if (error) {
>>                 sdev_printk(KERN_INFO, sdev,
>>                                 "failed to add device handler: %d\n", 
>> error);
>>                 return error;
>>         }
>>
>> to scsi_sysfs_add_sdev are adding a regression.
>>
>> 1. If that fails, then we forget to do device_del before doing the 
>> return. My patch in this thread added that back, so we do not see the 
>> sysfs oopses anymore. But.....
> 
> Ok.
> 
>> 2. It looks like in older kernels, we would allow misconfigured 
>> targets like this one to still setup devices. Do we want that old behavior
>> back?
>> Should we just ignore the return value from scsi_dh_add_device above?
>> Note that in this case, it is LIO so it can be easily fixed on the 
>> target side by just setting it up properly. I do not think other 
>> targets would hit this type of issue.
> 
> Be liberal in what you accept..  I guess we need to continue allowing 
> to connect to these broken targets, but a warning would be useful.
> 
> Can you send a patch?

Serguei, can you try the attached patch? Drop the other one I sent.

On Mon, 2016-02-01 at 10:55 -0600, Mike Christie wrote:
> On 01/30/2016 01:38 AM, Nicholas A. Bellinger wrote:
> > On Fri, 2016-01-29 at 17:32 -0600, Mike Christie wrote:
> >> On 01/29/2016 04:21 PM, Serguei Bezverkhi (sbezverk) wrote:
> >>> HI Mike,
> >>>
> >>> I tried your patch and it is has eliminated first traceback but I still
> do not see my remote targets.
> >>>
> >>
> >> That is sort of expected. Your target is not setup for ALUA properly. It
> >> says it supports ALUA, but when scsi_dh_alua asks about the ports it is
> >> reporting there are none. Ccing the people that made the patch that
> >> added the issue and own the code.
> >>
> >> Hey Christoph and Hannes,
> >>
> >> The dh/alua changes that added this:
> >>
> >>         error = scsi_dh_add_device(sdev);
> >>         if (error) {
> >>                 sdev_printk(KERN_INFO, sdev,
> >>                                 "failed to add device handler: %d\n",
> >> error);
> >>                 return error;
> >>         }
> >>
> >> to scsi_sysfs_add_sdev are adding a regression.
> >>
> >> 1. If that fails, then we forget to do device_del before doing the
> >> return. My patch in this thread added that back, so we do not see the
> >> sysfs oopses anymore. But.....
> >>
> >> 2. It looks like in older kernels, we would allow misconfigured targets
> >> like this one to still setup devices. Do we want that old behavior back?
> >> Should we just ignore the return value from scsi_dh_add_device above?
> >> Note that in this case, it is LIO so it can be easily fixed on the
> >> target side by just setting it up properly. I do not think other targets
> >> would hit this type of issue.
> >>
> > 
> > Btw, what does misconfigured mean here wrt target ALUA..?
> 
> [   25.833195] sd 6:0:0:4: alua: supports implicit and explicit TPGS
> [   25.833360] sd 6:0:0:4: alua: No target port descriptors found
> [   25.833363] sd 6:0:0:4: alua: Attach failed (-22)
> [   25.833365] sd 6:0:0:4: failed to add device handler: -22
> 

Strange, this hasn't changed in forever on the target side..

> He has LIO configured to report it supports implicit/explicit ALUA, but
> the ports do not seem to be configured.
> 
> For the LIO config side, are his LUNs just not in a the default_lu_gp or
> any other group?

So every non-PSCSI backend device becomes part of default_lu_gp +
default_tg_pt_gp and automatically shows up in EVPD=0x83, without user
needing to do any additional configuration.

Here's what the output looks like:

root@haakon3:/usr/src/target-pending.git# sg_inq -Hi /dev/sdb
VPD INQUIRY: Device Identification page
  <SNIP>
  Designation descriptor number 3, descriptor length: 8
    transport: Serial Attached SCSI Protocol (SPL-2)
    designator_type: Relative target port,  code_set: Binary
    associated with the target port
    designator header(hex): 61 94 00 04
    designator:
 00     00 00 00 02                                         ....
  Designation descriptor number 4, descriptor length: 8
    transport: Serial Attached SCSI Protocol (SPL-2)
    designator_type: Target port group,  code_set: Binary
    associated with the target port
    designator header(hex): 61 95 00 04
    designator:
 00     00 00 00 00                                         ....
  Designation descriptor number 5, descriptor length: 8
    designator_type: Logical unit group,  code_set: Binary
    associated with the addressed logical unit
    designator header(hex): 01 06 00 04
    designator:
 00     00 00 00 00                                         ....
 <SNIP>

So AFAICT, the relative target port, target port group, and logical unit
group being returned from target on v4.5-rc1 code looks correct.

Serguei, can you confirm with 'sg_inq -Hi /dev/sdX' output on your side
with the v3.10 based target..?

AFAICT the parsing in scsi_vpd_tpg_id() from commit a8aa3978 looks
correct too.

Hannes, any ideas..?

Here you go, same output to compare for 4.4.1 and 3.10.0

Linux 4.4.1 #1 SMP Tue Feb 2 16:15:36 EST 2016

[root@sbezverk-osp-3 ~(keystone_admin)]#  sg_inq -Hi /dev/sdc
VPD INQUIRY: Device Identification page
  Designation descriptor number 1, descriptor length: 20
    designator_type: NAA,  code_set: Binary
    associated with the addressed logical unit
    designator header(hex): 01 03 00 10
    designator:
 00     60 01 40 56 5f e9 25 e7  94 a4 20 69 2c 0c b3 c6    `.@V_.%... i,...
  Designation descriptor number 2, descriptor length: 60
    designator_type: T10 vendor identification,  code_set: ASCII
    associated with the addressed logical unit
    designator header(hex): 02 01 00 38
    designator:
 00     4c 49 4f 2d 4f 52 47 00  73 61 6e 2d 64 69 73 6b    LIO-ORG.san-disk
 10     2d 32 3a 36 35 66 65 39  32 35 65 2d 37 39 34 61    -2:65fe925e-794a
 20     2d 34 32 30 36 2d 39 32  63 30 2d 63 62 33 63 36    -4206-92c0-cb3c6
 30     39 61 35 33 34 39 61 00                             9a5349a.
  Designation descriptor number 3, descriptor length: 8
    transport: Internet SCSI (iSCSI)
    designator_type: Relative target port,  code_set: Binary
    associated with the target port
    designator header(hex): 51 94 00 04
    designator:
 00     00 00 00 01                                         ....
  Designation descriptor number 4, descriptor length: 8
    transport: Internet SCSI (iSCSI)
    designator_type: Target port group,  code_set: Binary
    associated with the target port
    designator header(hex): 51 95 00 04
    designator:
 00     00 00 00 00                                         ....
  Designation descriptor number 5, descriptor length: 8
    designator_type: Logical unit group,  code_set: Binary
    associated with the addressed logical unit
    designator header(hex): 01 06 00 04
    designator:
 00     00 00 00 00                                         ....
  Designation descriptor number 6, descriptor length: 80
    transport: Internet SCSI (iSCSI)
    designator_type: SCSI name string,  code_set: UTF-8
    associated with the target port
    designator header(hex): 53 98 00 4c
    designator:
 00     69 71 6e 2e 32 30 30 33  2d 30 31 2e 6f 72 67 2e    iqn.2003-01.org.
 10     6c 69 6e 75 78 2d 69 73  63 73 69 2e 73 62 65 7a    linux-iscsi.sbez
 20     76 65 72 6b 2d 73 61 6e  2d 31 2e 78 38 36 36 34    verk-san-1.x8664
 30     3a 73 6e 2e 33 64 66 63  66 66 62 64 66 66 34 33    :sn.3dfcffbdff43
 40     2c 74 2c 30 78 30 30 30  31 00 00 00                ,t,0x0001...
  Designation descriptor number 7, descriptor length: 72
    transport: Internet SCSI (iSCSI)
    designator_type: SCSI name string,  code_set: UTF-8
    associated with the target device that contains addressed lu
    designator header(hex): 53 a8 00 44
    designator:
 00     69 71 6e 2e 32 30 30 33  2d 30 31 2e 6f 72 67 2e    iqn.2003-01.org.
 10     6c 69 6e 75 78 2d 69 73  63 73 69 2e 73 62 65 7a    linux-iscsi.sbez
 20     76 65 72 6b 2d 73 61 6e  2d 31 2e 78 38 36 36 34    verk-san-1.x8664
 30     3a 73 6e 2e 33 64 66 63  66 66 62 64 66 66 34 33    :sn.3dfcffbdff43
 40     00 00 00 00

Linux 3.10.0-327.4.5.el7.x86_64 #1 SMP Thu Jan 21 04:10:29 EST 2016

 [root@sbezverk-osp-3 ~(keystone_admin)]# sg_inq -Hi /dev/sdc

VPD INQUIRY: Device Identification page
  Designation descriptor number 1, descriptor length: 20
    designator_type: NAA,  code_set: Binary
    associated with the addressed logical unit
    designator header(hex): 01 03 00 10
    designator:
 00     60 01 40 56 5f e9 25 e7  94 a4 20 69 2c 0c b3 c6    `.@V_.%... i,...
  Designation descriptor number 2, descriptor length: 60
    designator_type: T10 vendor identification,  code_set: ASCII
    associated with the addressed logical unit
    designator header(hex): 02 01 00 38
    designator:
 00     4c 49 4f 2d 4f 52 47 00  73 61 6e 2d 64 69 73 6b    LIO-ORG.san-disk
 10     2d 32 3a 36 35 66 65 39  32 35 65 2d 37 39 34 61    -2:65fe925e-794a
 20     2d 34 32 30 36 2d 39 32  63 30 2d 63 62 33 63 36    -4206-92c0-cb3c6
 30     39 61 35 33 34 39 61 00                             9a5349a.
  Designation descriptor number 3, descriptor length: 8
    transport: Internet SCSI (iSCSI)
    designator_type: Relative target port,  code_set: Binary
    associated with the target port
    designator header(hex): 51 94 00 04
    designator:
 00     00 00 00 01                                         ....
  Designation descriptor number 4, descriptor length: 8
    transport: Internet SCSI (iSCSI)
    designator_type: Target port group,  code_set: Binary
    associated with the target port
    designator header(hex): 51 95 00 04
    designator:
 00     00 00 00 00                                         ....
  Designation descriptor number 5, descriptor length: 8
    designator_type: Logical unit group,  code_set: Binary
    associated with the addressed logical unit
    designator header(hex): 01 06 00 04
    designator:
 00     00 00 00 00                                         ....
  Designation descriptor number 6, descriptor length: 80
    transport: Internet SCSI (iSCSI)
    designator_type: SCSI name string,  code_set: UTF-8
    associated with the target port
    designator header(hex): 53 98 00 4c
    designator:
 00     69 71 6e 2e 32 30 30 33  2d 30 31 2e 6f 72 67 2e    iqn.2003-01.org.
 10     6c 69 6e 75 78 2d 69 73  63 73 69 2e 73 62 65 7a    linux-iscsi.sbez
 20     76 65 72 6b 2d 73 61 6e  2d 31 2e 78 38 36 36 34    verk-san-1.x8664
 30     3a 73 6e 2e 33 64 66 63  66 66 62 64 66 66 34 33    :sn.3dfcffbdff43
 40     2c 74 2c 30 78 30 30 30  31 00 00 00                ,t,0x0001...
  Designation descriptor number 7, descriptor length: 72
    transport: Internet SCSI (iSCSI)
    designator_type: SCSI name string,  code_set: UTF-8
    associated with the target device that contains addressed lu
    designator header(hex): 53 a8 00 44
    designator:
 00     69 71 6e 2e 32 30 30 33  2d 30 31 2e 6f 72 67 2e    iqn.2003-01.org.
 10     6c 69 6e 75 78 2d 69 73  63 73 69 2e 73 62 65 7a    linux-iscsi.sbez
 20     76 65 72 6b 2d 73 61 6e  2d 31 2e 78 38 36 36 34    verk-san-1.x8664
 30     3a 73 6e 2e 33 64 66 63  66 66 62 64 66 66 34 33    :sn.3dfcffbdff43
 40     00 00 00 00                           


Let me know if you need any additional info.

Thank you

Serguei


-----Original Message-----
From: Nicholas A. Bellinger [mailto:nab@linux-iscsi.org] 
Sent: Tuesday, February 02, 2016 5:56 PM
To: Mike Christie <michaelc@cs.wisc.edu>
Cc: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>; bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org; Christoph Hellwig <hch@infradead.org>; Hannes Reinecke <hare@suse.de>
Subject: Re: [Bug 111441] New: iscsi fails to attach to targets

On Mon, 2016-02-01 at 10:55 -0600, Mike Christie wrote:
> On 01/30/2016 01:38 AM, Nicholas A. Bellinger wrote:
> > On Fri, 2016-01-29 at 17:32 -0600, Mike Christie wrote:
> >> On 01/29/2016 04:21 PM, Serguei Bezverkhi (sbezverk) wrote:
> >>> HI Mike,
> >>>
> >>> I tried your patch and it is has eliminated first traceback but I still
> do not see my remote targets.
> >>>
> >>
> >> That is sort of expected. Your target is not setup for ALUA 
> >> properly. It says it supports ALUA, but when scsi_dh_alua asks 
> >> about the ports it is reporting there are none. Ccing the people 
> >> that made the patch that added the issue and own the code.
> >>
> >> Hey Christoph and Hannes,
> >>
> >> The dh/alua changes that added this:
> >>
> >>         error = scsi_dh_add_device(sdev);
> >>         if (error) {
> >>                 sdev_printk(KERN_INFO, sdev,
> >>                                 "failed to add device handler: 
> >> %d\n", error);
> >>                 return error;
> >>         }
> >>
> >> to scsi_sysfs_add_sdev are adding a regression.
> >>
> >> 1. If that fails, then we forget to do device_del before doing the 
> >> return. My patch in this thread added that back, so we do not see 
> >> the sysfs oopses anymore. But.....
> >>
> >> 2. It looks like in older kernels, we would allow misconfigured 
> >> targets like this one to still setup devices. Do we want that old behavior
> back?
> >> Should we just ignore the return value from scsi_dh_add_device above?
> >> Note that in this case, it is LIO so it can be easily fixed on the 
> >> target side by just setting it up properly. I do not think other 
> >> targets would hit this type of issue.
> >>
> > 
> > Btw, what does misconfigured mean here wrt target ALUA..?
> 
> [   25.833195] sd 6:0:0:4: alua: supports implicit and explicit TPGS
> [   25.833360] sd 6:0:0:4: alua: No target port descriptors found
> [   25.833363] sd 6:0:0:4: alua: Attach failed (-22)
> [   25.833365] sd 6:0:0:4: failed to add device handler: -22
> 

Strange, this hasn't changed in forever on the target side..

> He has LIO configured to report it supports implicit/explicit ALUA, 
> but the ports do not seem to be configured.
> 
> For the LIO config side, are his LUNs just not in a the default_lu_gp 
> or any other group?

So every non-PSCSI backend device becomes part of default_lu_gp + default_tg_pt_gp and automatically shows up in EVPD=0x83, without user needing to do any additional configuration.

Here's what the output looks like:

root@haakon3:/usr/src/target-pending.git# sg_inq -Hi /dev/sdb VPD INQUIRY: Device Identification page
  <SNIP>
  Designation descriptor number 3, descriptor length: 8
    transport: Serial Attached SCSI Protocol (SPL-2)
    designator_type: Relative target port,  code_set: Binary
    associated with the target port
    designator header(hex): 61 94 00 04
    designator:
 00     00 00 00 02                                         ....
  Designation descriptor number 4, descriptor length: 8
    transport: Serial Attached SCSI Protocol (SPL-2)
    designator_type: Target port group,  code_set: Binary
    associated with the target port
    designator header(hex): 61 95 00 04
    designator:
 00     00 00 00 00                                         ....
  Designation descriptor number 5, descriptor length: 8
    designator_type: Logical unit group,  code_set: Binary
    associated with the addressed logical unit
    designator header(hex): 01 06 00 04
    designator:
 00     00 00 00 00                                         ....
 <SNIP>

So AFAICT, the relative target port, target port group, and logical unit group being returned from target on v4.5-rc1 code looks correct.

Serguei, can you confirm with 'sg_inq -Hi /dev/sdX' output on your side with the v3.10 based target..?

AFAICT the parsing in scsi_vpd_tpg_id() from commit a8aa3978 looks correct too.

Hannes, any ideas..?

On Tue, 2016-02-02 at 14:56 -0800, Nicholas A. Bellinger wrote:
> On Mon, 2016-02-01 at 10:55 -0600, Mike Christie wrote:
> > On 01/30/2016 01:38 AM, Nicholas A. Bellinger wrote:
> > > On Fri, 2016-01-29 at 17:32 -0600, Mike Christie wrote:
> > >> On 01/29/2016 04:21 PM, Serguei Bezverkhi (sbezverk) wrote:
> > >>> HI Mike,
> > >>>
> > >>> I tried your patch and it is has eliminated first traceback but I still
> do not see my remote targets.
> > >>>
> > >>
> > >> That is sort of expected. Your target is not setup for ALUA properly. It
> > >> says it supports ALUA, but when scsi_dh_alua asks about the ports it is
> > >> reporting there are none. Ccing the people that made the patch that
> > >> added the issue and own the code.
> > >>
> > >> Hey Christoph and Hannes,
> > >>
> > >> The dh/alua changes that added this:
> > >>
> > >>         error = scsi_dh_add_device(sdev);
> > >>         if (error) {
> > >>                 sdev_printk(KERN_INFO, sdev,
> > >>                                 "failed to add device handler: %d\n",
> > >> error);
> > >>                 return error;
> > >>         }
> > >>
> > >> to scsi_sysfs_add_sdev are adding a regression.
> > >>
> > >> 1. If that fails, then we forget to do device_del before doing the
> > >> return. My patch in this thread added that back, so we do not see the
> > >> sysfs oopses anymore. But.....
> > >>
> > >> 2. It looks like in older kernels, we would allow misconfigured targets
> > >> like this one to still setup devices. Do we want that old behavior back?
> > >> Should we just ignore the return value from scsi_dh_add_device above?
> > >> Note that in this case, it is LIO so it can be easily fixed on the
> > >> target side by just setting it up properly. I do not think other targets
> > >> would hit this type of issue.
> > >>
> > > 
> > > Btw, what does misconfigured mean here wrt target ALUA..?
> > 
> > [   25.833195] sd 6:0:0:4: alua: supports implicit and explicit TPGS
> > [   25.833360] sd 6:0:0:4: alua: No target port descriptors found
> > [   25.833363] sd 6:0:0:4: alua: Attach failed (-22)
> > [   25.833365] sd 6:0:0:4: failed to add device handler: -22
> > 
> 
> Strange, this hasn't changed in forever on the target side..
> 
> > He has LIO configured to report it supports implicit/explicit ALUA, but
> > the ports do not seem to be configured.
> > 
> > For the LIO config side, are his LUNs just not in a the default_lu_gp or
> > any other group?
> 
> So every non-PSCSI backend device becomes part of default_lu_gp +
> default_tg_pt_gp and automatically shows up in EVPD=0x83, without user
> needing to do any additional configuration.
> 
> Here's what the output looks like:
> 
> root@haakon3:/usr/src/target-pending.git# sg_inq -Hi /dev/sdb
> VPD INQUIRY: Device Identification page
>   <SNIP>
>   Designation descriptor number 3, descriptor length: 8
>     transport: Serial Attached SCSI Protocol (SPL-2)
>     designator_type: Relative target port,  code_set: Binary
>     associated with the target port
>     designator header(hex): 61 94 00 04
>     designator:
>  00     00 00 00 02                                         ....
>   Designation descriptor number 4, descriptor length: 8
>     transport: Serial Attached SCSI Protocol (SPL-2)
>     designator_type: Target port group,  code_set: Binary
>     associated with the target port
>     designator header(hex): 61 95 00 04
>     designator:
>  00     00 00 00 00                                         ....
>   Designation descriptor number 5, descriptor length: 8
>     designator_type: Logical unit group,  code_set: Binary
>     associated with the addressed logical unit
>     designator header(hex): 01 06 00 04
>     designator:
>  00     00 00 00 00                                         ....
>  <SNIP>
> 
> So AFAICT, the relative target port, target port group, and logical unit
> group being returned from target on v4.5-rc1 code looks correct.
> 
> Serguei, can you confirm with 'sg_inq -Hi /dev/sdX' output on your side
> with the v3.10 based target..?
> 
> AFAICT the parsing in scsi_vpd_tpg_id() from commit a8aa3978 looks
> correct too.
> 
> Hannes, any ideas..?

Ping.

Hi Mike,

I just wanted to follow up with you to see if the patch got committed to an upstream kernel if yes, please let me into which version it went.

Thank you

Serguei


Serguei Bezverkhi,
TECHNICAL LEADER.SERVICES
Global SP Services
sbezverk@cisco.com
Phone: +1 416 306 7312
Mobile: +1 514 234 7374

CCIE (R&S,SP,Sec) - #9527

Cisco.com



 Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here for Company Registration Information.



-----Original Message-----
From: Mike Christie [mailto:michaelc@cs.wisc.edu] 
Sent: Friday, January 29, 2016 6:33 PM
To: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>
Cc: bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org; Christoph Hellwig <hch@infradead.org>; Hannes Reinecke <hare@suse.de>
Subject: Re: [Bug 111441] New: iscsi fails to attach to targets

On 01/29/2016 04:21 PM, Serguei Bezverkhi (sbezverk) wrote:
> HI Mike,
> 
> I tried your patch and it is has eliminated first traceback but I still do
> not see my remote targets.
> 

That is sort of expected. Your target is not setup for ALUA properly. It says it supports ALUA, but when scsi_dh_alua asks about the ports it is reporting there are none. Ccing the people that made the patch that added the issue and own the code.

Hey Christoph and Hannes,

The dh/alua changes that added this:

        error = scsi_dh_add_device(sdev);
        if (error) {
                sdev_printk(KERN_INFO, sdev,
                                "failed to add device handler: %d\n", error);
                return error;
        }

to scsi_sysfs_add_sdev are adding a regression.

1. If that fails, then we forget to do device_del before doing the return. My patch in this thread added that back, so we do not see the sysfs oopses anymore. But.....

2. It looks like in older kernels, we would allow misconfigured targets like this one to still setup devices. Do we want that old behavior back?
Should we just ignore the return value from scsi_dh_add_device above?
Note that in this case, it is LIO so it can be easily fixed on the target side by just setting it up properly. I do not think other targets would hit this type of issue.





> 
> Here is dmesg
> 
> [   26.103812] scsi 3:0:0:2: Direct-Access     LIO-ORG  san-disk-2       4.0 
> PQ: 0 ANSI: 5
> [   26.104338] sd 3:0:0:2: alua: supports implicit and explicit TPGS
> [   26.104549] sd 3:0:0:2: alua: No target port descriptors found
> [   26.104552] sd 3:0:0:2: alua: Attach failed (-22)
> [   26.104554] sd 3:0:0:2: failed to add device handler: -22
> [   26.104578] sd 3:0:0:2: [sdc] 20507809792 512-byte logical blocks: (10.4
> TB/9.54 TiB)
> [   26.104905] sd 3:0:0:2: [sdc] Write Protect is off
> [   26.104908] sd 3:0:0:2: [sdc] Mode Sense: 43 00 10 08
> [   26.105036] sd 3:0:0:2: [sdc] Write cache: enabled, read cache: enabled,
> supports DPO and FUA
> [   26.112294] scsi host6: iSCSI Initiator over TCP/IP
> [   26.113279] scsi 4:0:0:3: Direct-Access     LIO-ORG  san-disk-3       4.0 
> PQ: 0 ANSI: 5
> [   26.113690] sd 4:0:0:3: alua: supports implicit and explicit TPGS
> [   26.113877] sd 4:0:0:3: [sdd] 9765625856 512-byte logical blocks: (5.00
> TB/4.54 TiB)
> [   26.113948] sd 4:0:0:3: alua: No target port descriptors found
> [   26.113951] sd 4:0:0:3: alua: Attach failed (-22)
> [   26.113953] sd 4:0:0:3: failed to add device handler: -22
> [   26.114292] sd 4:0:0:3: [sdd] Write Protect is off
> [   26.114295] sd 4:0:0:3: [sdd] Mode Sense: 43 00 10 08
> [   26.114503] sd 4:0:0:3: [sdd] Write cache: enabled, read cache: enabled,
> supports DPO and FUA
> [   26.123875] scsi 5:0:0:1: Direct-Access     LIO-ORG  san-disk-1       4.0 
> PQ: 0 ANSI: 5
> [   26.123911] scsi 6:0:0:4: Direct-Access     LIO-ORG  san-disk-4       4.0 
> PQ: 0 ANSI: 5
> [   26.124452] sd 6:0:0:4: alua: supports implicit and explicit TPGS
> [   26.124453] sd 5:0:0:1: alua: supports implicit and explicit TPGS
> [   26.124724] sd 5:0:0:1: alua: No target port descriptors found
> [   26.124727] sd 5:0:0:1: alua: Attach failed (-22)
> [   26.124728] sd 5:0:0:1: failed to add device handler: -22
> [   26.124736] sd 6:0:0:4: [sde] 10742171648 512-byte logical blocks: (5.49
> TB/5.00 TiB)
> [   26.124773] sd 5:0:0:1: [sdf] 7812499389 512-byte logical blocks: (3.99
> TB/3.63 TiB)
> [   26.124777] sd 6:0:0:4: alua: No target port descriptors found
> [   26.124779] sd 6:0:0:4: alua: Attach failed (-22)
> [   26.124780] sd 6:0:0:4: failed to add device handler: -22
> [   26.125182] sd 5:0:0:1: [sdf] Write Protect is off
> [   26.125184] sd 5:0:0:1: [sdf] Mode Sense: 43 00 10 08
> [   26.125217] sd 6:0:0:4: [sde] Write Protect is off
> [   26.125220] sd 6:0:0:4: [sde] Mode Sense: 43 00 10 08
> [   26.125306] sd 5:0:0:1: [sdf] Write cache: enabled, read cache: enabled,
> supports DPO and FUA
> [   26.125512] sd 6:0:0:4: [sde] Write cache: enabled, read cache: enabled,
> supports DPO and FUA
> [   26.129633]  sdf: sdf1
> [   26.130637] sd 5:0:0:1: [sdf] Attached SCSI disk
> [   26.144377] ixgbe 0000:04:00.0: registered PHC device on enp4s0f0
> [   26.149072]  sdc: sdc1
> [   26.150434] sd 3:0:0:2: [sdc] Attached SCSI disk
> [   26.190709]  sdd: sdd1 sdd2
> [   26.193348] sd 4:0:0:3: [sdd] Attached SCSI disk
> [   26.230515]  sde: sde1
> [   26.231674] sd 6:0:0:4: [sde] Attached SCSI disk
> [   26.231987] sd 6:0:0:4: [sde] Synchronizing SCSI cache
> [   26.232021] sd 5:0:0:1: [sdf] Synchronizing SCSI cache
> [   26.233212] sd 3:0:0:2: [sdc] Synchronizing SCSI cache
> [   26.233440] sd 4:0:0:3: [sdd] Synchronizing SCSI cache
> [   26.236755] Buffer I/O error on dev sdc, logical block 2563476132, async
> page read
> [   26.238897] Buffer I/O error on dev sdd, logical block 1220703182, async
> page read
> [   26.245773] ixgbe 0000:04:00.1: SR-IOV enabled with 8 VFs
> [   26.245775] ixgbe 0000:04:00.1: configure port vlans to keep your VFs
> secure
> [   26.274544] scsi 6:0:0:0: Unexpected response from lun 4 while scanning,
> scan aborted
> [   26.283173] scsi 3:0:0:0: Unexpected response from lun 2 while scanning,
> scan aborted
> [   26.288571] scsi 4:0:0:0: Unexpected response from lun 3 while scanning,
> scan aborted
> [   26.288618] scsi 5:0:0:0: Unexpected response from lun 1 while scanning,
> scan aborted
> 
> 
> Second traceback is gone too, but still no luck attaching local iscsi targets
> either.
> 
> 
> [  639.148875] TARGET_CORE[iSCSI]: Expected Transfer Length: 264 does 
> not match SCSI CDB Length: 8 for SAM Opcode: 0x12 [  639.148911] sd 
> 7:0:0:0: [sdc] 115343360 512-byte logical blocks: (59.0 GB/55.0 GiB) [  
> 639.148925] sd 7:0:0:0: alua: No target port descriptors found [  
> 639.148928] sd 7:0:0:0: alua: Attach failed (-22) [  639.149186] sd 
> 7:0:0:0: [sdc] Write Protect is off [  639.149188] sd 7:0:0:0: [sdc] 
> Mode Sense: 43 00 10 08 [  639.149279] sd 7:0:0:0: [sdc] Write cache: 
> enabled, read cache: enabled, supports DPO and FUA [  639.149298]
> iSCSI/iqn.1994-05.com.redhat:cf7f1fafca4b: Unsupported SCSI Opcode 0xa3,
> sending CHECK_CONDITION.
> [  639.149530] sd 7:0:0:0: failed to add device handler: -22 [  
> 639.154762] sd 7:0:0:0: [sdc] Attached SCSI disk [  639.154857] sd 
> 7:0:0:0: [sdc] Synchronizing SCSI cache
> [  655.279047] scsi 7:0:0:0: Direct-Access     LIO-ORG  IBLOCK           4.0 
> PQ: 0 ANSI: 5
> [  655.279397] sd 7:0:0:0: alua: supports implicit and explicit TPGS [  
> 655.279503] TARGET_CORE[iSCSI]: Expected Transfer Length: 264 does not 
> match SCSI CDB Length: 8 for SAM Opcode: 0x12 [  655.279533] sd 
> 7:0:0:0: alua: No target port descriptors found [  655.279535] sd 
> 7:0:0:0: alua: Attach failed (-22) [  655.279587] sd 7:0:0:0: [sdc] 
> 115343360 512-byte logical blocks: (59.0 GB/55.0 GiB) [  655.279848] 
> sd 7:0:0:0: [sdc] Write Protect is off [  655.279849] sd 7:0:0:0: 
> [sdc] Mode Sense: 43 00 10 08 [  655.279981] sd 7:0:0:0: [sdc] Write 
> cache: enabled, read cache: enabled, supports DPO and FUA [  655.280034]
> iSCSI/iqn.1994-05.com.redhat:cf7f1fafca4b: Unsupported SCSI Opcode 0xa3,
> sending CHECK_CONDITION.
> [  655.280171] sd 7:0:0:0: failed to add device handler: -22 [  
> 655.286008] sd 7:0:0:0: [sdc] Attached SCSI disk [  655.286132] sd 
> 7:0:0:0: [sdc] Synchronizing SCSI cache
> 
> 
> Serguei Bezverkhi,
> TECHNICAL LEADER.SERVICES
> Global SP Services
> sbezverk@cisco.com
> Phone: +1 416 306 7312
> Mobile: +1 514 234 7374
> 
> CCIE (R&S,SP,Sec) - #9527
> 
> Cisco.com
> 
> 
> 
>  Think before you print.
> This email may contain confidential and privileged material for the sole use
> of the intended recipient. Any review, use, distribution or disclosure by
> others is strictly prohibited. If you are not the intended recipient (or
> authorized to receive for the recipient), please contact the sender by reply
> email and delete all copies of this message.
> Please click here for Company Registration Information.
> 
> 
> 
> 
> -----Original Message-----
> From: Mike Christie [mailto:michaelc@cs.wisc.edu]
> Sent: Friday, January 29, 2016 2:27 PM
> To: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>
> Cc: bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org
> Subject: Re: [Bug 111441] New: iscsi fails to attach to targets
> 
> 
> 
> On 01/29/2016 01:11 PM, Serguei Bezverkhi (sbezverk) wrote:
>> If you send me the diff for your patch, I will build new kernel myself.
>>
> 
> Bugzilla must be messing something up. I attached to one of the previous
> mails. Attaching it here again.
> 
> Email me offlist and without bugzilla if you do not get it here.
> 
> The patch will fix the syfs bug ons you are hitting.
> 
> I am not sure if it will fix the genhd one. We can deal with that one next if
> it is a different issue.
> 
> 
>> Serguei
>>
>>
>> Serguei Bezverkhi,
>> TECHNICAL LEADER.SERVICES
>> Global SP Services
>> sbezverk@cisco.com
>> Phone: +1 416 306 7312
>> Mobile: +1 514 234 7374
>>
>> CCIE (R&S,SP,Sec) - #9527
>>
>> Cisco.com
>>
>>
>>
>>  Think before you print.
>> This email may contain confidential and privileged material for the sole use
>> of the intended recipient. Any review, use, distribution or disclosure by
>> others is strictly prohibited. If you are not the intended recipient (or
>> authorized to receive for the recipient), please contact the sender by reply
>> email and delete all copies of this message.
>> Please click here for Company Registration Information.
>>
>>
>>
>>
>> -----Original Message-----
>> From: Michael Christie [mailto:michaelc@cs.wisc.edu]
>> Sent: Friday, January 29, 2016 2:09 PM
>> To: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>
>> Cc: bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org
>> Subject: Re: [Bug 111441] New: iscsi fails to attach to targets
>>
>>
>>> On Jan 29, 2016, at 6:04 AM, Serguei Bezverkhi (sbezverk)
>>> <sbezverk@cisco.com> wrote:
>>>
>>> Actually this server uses both cases: Local taregts (since it is OpenStack
>>> server) and remote targets as it tries to mount 4 remotefile systems.  
>>>
>>> You are correct, I always use the same box I just change the kernel it is
>>> using to boot. No other changes to the environment. I do not mind to load a
>>> test kernel without that suspected patch, just get me the RPM.
>>>
>>
>> I do not know what you mean. I think the patch I sent will fix the sysfs
>> errors caused due to alua not being setup properly on your system and
>> scsi_dh_alua failing to attach. That patch should be applied to the 4.4
>> upstream kernel. Are you saying you want me to make you a kernel rpm?
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-scsi" 
>> in the body of a message to majordomo@vger.kernel.org More majordomo 
>> info at  http://vger.kernel.org/majordomo-info.html
>>
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" 
> in the body of a message to majordomo@vger.kernel.org More majordomo 
> info at  http://vger.kernel.org/majordomo-info.html
>

On 02/22/2016 01:45 AM, Serguei Bezverkhi (sbezverk) wrote:
> Hi Mike,
> 
> I just wanted to follow up with you to see if the patch got committed to an
> upstream kernel if yes, please let me into which version it went.
> 
> Thank you
> 
> Serguei
> 
> 
> Serguei Bezverkhi,
> TECHNICAL LEADER.SERVICES
> Global SP Services
> sbezverk@cisco.com
> Phone: +1 416 306 7312
> Mobile: +1 514 234 7374
> 
> CCIE (R&S,SP,Sec) - #9527
> 
> Cisco.com
> 
> 
> 
>  Think before you print.
> This email may contain confidential and privileged material for the sole use
> of the intended recipient. Any review, use, distribution or disclosure by
> others is strictly prohibited. If you are not the intended recipient (or
> authorized to receive for the recipient), please contact the sender by reply
> email and delete all copies of this message.
> Please click here for Company Registration Information.
> 
> 
> 
> -----Original Message-----
> From: Mike Christie [mailto:michaelc@cs.wisc.edu] 
> Sent: Friday, January 29, 2016 6:33 PM
> To: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>
> Cc: bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org;
> Christoph Hellwig <hch@infradead.org>; Hannes Reinecke <hare@suse.de>
> Subject: Re: [Bug 111441] New: iscsi fails to attach to targets
> 
> On 01/29/2016 04:21 PM, Serguei Bezverkhi (sbezverk) wrote:
>> HI Mike,
>>
>> I tried your patch and it is has eliminated first traceback but I still do
>> not see my remote targets.
>>
> 
> That is sort of expected. Your target is not setup for ALUA properly. It says
> it supports ALUA, but when scsi_dh_alua asks about the ports it is reporting
> there are none. Ccing the people that made the patch that added the issue and
> own the code.
> 
> Hey Christoph and Hannes,
> 
> The dh/alua changes that added this:
> 
>         error = scsi_dh_add_device(sdev);
>         if (error) {
>                 sdev_printk(KERN_INFO, sdev,
>                                 "failed to add device handler: %d\n", error);
>                 return error;
>         }
> 
> to scsi_sysfs_add_sdev are adding a regression.
> 
> 1. If that fails, then we forget to do device_del before doing the return. My
> patch in this thread added that back, so we do not see the sysfs oopses
> anymore. But.....
> 
> 2. It looks like in older kernels, we would allow misconfigured targets like
> this one to still setup devices. Do we want that old behavior back?
> Should we just ignore the return value from scsi_dh_add_device above?
> Note that in this case, it is LIO so it can be easily fixed on the target
> side by just setting it up properly. I do not think other targets would hit
> this type of issue.
> 
> 
This has been fixed up with my patchset to update the ALUA handler, most
notably the commit 'scsi: ignore errors from scsi_dh_add_device()' which
was included in 4.5.

Cheers,

Hannes

Hello Hannes,

Thank you for your reply. I am on 4.4.2 kernel, is there any chance to commit it in 4.4 as well? If not, could you send me diff for 4.4 kernel.

Best regards

Serguei


Serguei Bezverkhi,
TECHNICAL LEADER.SERVICES
Global SP Services
sbezverk@cisco.com
Phone: +1 416 306 7312
Mobile: +1 514 234 7374

CCIE (R&S,SP,Sec) - #9527

Cisco.com



 Think before you print.
This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.
Please click here for Company Registration Information.




-----Original Message-----
From: Hannes Reinecke [mailto:hare@suse.de] 
Sent: Monday, February 22, 2016 2:08 AM
To: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>; Mike Christie <michaelc@cs.wisc.edu>
Cc: bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org; Christoph Hellwig <hch@infradead.org>
Subject: Re: [Bug 111441] New: iscsi fails to attach to targets

On 02/22/2016 01:45 AM, Serguei Bezverkhi (sbezverk) wrote:
> Hi Mike,
> 
> I just wanted to follow up with you to see if the patch got committed to an
> upstream kernel if yes, please let me into which version it went.
> 
> Thank you
> 
> Serguei
> 
> 
> Serguei Bezverkhi,
> TECHNICAL LEADER.SERVICES
> Global SP Services
> sbezverk@cisco.com
> Phone: +1 416 306 7312
> Mobile: +1 514 234 7374
> 
> CCIE (R&S,SP,Sec) - #9527
> 
> Cisco.com
> 
> 
> 
>  Think before you print.
> This email may contain confidential and privileged material for the sole use
> of the intended recipient. Any review, use, distribution or disclosure by
> others is strictly prohibited. If you are not the intended recipient (or
> authorized to receive for the recipient), please contact the sender by reply
> email and delete all copies of this message.
> Please click here for Company Registration Information.
> 
> 
> 
> -----Original Message-----
> From: Mike Christie [mailto:michaelc@cs.wisc.edu]
> Sent: Friday, January 29, 2016 6:33 PM
> To: Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com>
> Cc: bugzilla-daemon@bugzilla.kernel.org; linux-scsi@vger.kernel.org; 
> Christoph Hellwig <hch@infradead.org>; Hannes Reinecke <hare@suse.de>
> Subject: Re: [Bug 111441] New: iscsi fails to attach to targets
> 
> On 01/29/2016 04:21 PM, Serguei Bezverkhi (sbezverk) wrote:
>> HI Mike,
>>
>> I tried your patch and it is has eliminated first traceback but I still do
>> not see my remote targets.
>>
> 
> That is sort of expected. Your target is not setup for ALUA properly. It says
> it supports ALUA, but when scsi_dh_alua asks about the ports it is reporting
> there are none. Ccing the people that made the patch that added the issue and
> own the code.
> 
> Hey Christoph and Hannes,
> 
> The dh/alua changes that added this:
> 
>         error = scsi_dh_add_device(sdev);
>         if (error) {
>                 sdev_printk(KERN_INFO, sdev,
>                                 "failed to add device handler: %d\n", error);
>                 return error;
>         }
> 
> to scsi_sysfs_add_sdev are adding a regression.
> 
> 1. If that fails, then we forget to do device_del before doing the return. My
> patch in this thread added that back, so we do not see the sysfs oopses
> anymore. But.....
> 
> 2. It looks like in older kernels, we would allow misconfigured targets like
> this one to still setup devices. Do we want that old behavior back?
> Should we just ignore the return value from scsi_dh_add_device above?
> Note that in this case, it is LIO so it can be easily fixed on the target
> side by just setting it up properly. I do not think other targets would hit
> this type of issue.
> 
> 
This has been fixed up with my patchset to update the ALUA handler, most notably the commit 'scsi: ignore errors from scsi_dh_add_device()' which was included in 4.5.

Cheers,

Hannes

Hey Hannes,

On Tue, 2016-02-16 at 20:08 +0100, Hannes Reinecke wrote:
> On 02/08/2016 09:01 AM, Nicholas A. Bellinger wrote:
> > On Tue, 2016-02-02 at 14:56 -0800, Nicholas A. Bellinger wrote:
> >> On Mon, 2016-02-01 at 10:55 -0600, Mike Christie wrote:
> >>> On 01/30/2016 01:38 AM, Nicholas A. Bellinger wrote:
> >>>> On Fri, 2016-01-29 at 17:32 -0600, Mike Christie wrote:

<SNIP>

> >>>> Btw, what does misconfigured mean here wrt target ALUA..?
> >>>
> >>> [   25.833195] sd 6:0:0:4: alua: supports implicit and explicit TPGS
> >>> [   25.833360] sd 6:0:0:4: alua: No target port descriptors found
> >>> [   25.833363] sd 6:0:0:4: alua: Attach failed (-22)
> >>> [   25.833365] sd 6:0:0:4: failed to add device handler: -22
> >>>
> >>
> >> Strange, this hasn't changed in forever on the target side..
> >>
> >>> He has LIO configured to report it supports implicit/explicit ALUA, but
> >>> the ports do not seem to be configured.
> >>>
> >>> For the LIO config side, are his LUNs just not in a the default_lu_gp or
> >>> any other group?
> >>
> >> So every non-PSCSI backend device becomes part of default_lu_gp +
> >> default_tg_pt_gp and automatically shows up in EVPD=0x83, without user
> >> needing to do any additional configuration.
> >>
> >> Here's what the output looks like:
> >>
> >> root@haakon3:/usr/src/target-pending.git# sg_inq -Hi /dev/sdb
> >> VPD INQUIRY: Device Identification page
> >>    <SNIP>
> >>    Designation descriptor number 3, descriptor length: 8
> >>      transport: Serial Attached SCSI Protocol (SPL-2)
> >>      designator_type: Relative target port,  code_set: Binary
> >>      associated with the target port
> >>      designator header(hex): 61 94 00 04
> >>      designator:
> >>   00     00 00 00 02                                         ....
> >>    Designation descriptor number 4, descriptor length: 8
> >>      transport: Serial Attached SCSI Protocol (SPL-2)
> >>      designator_type: Target port group,  code_set: Binary
> >>      associated with the target port
> >>      designator header(hex): 61 95 00 04
> >>      designator:
> >>   00     00 00 00 00                                         ....
> >>    Designation descriptor number 5, descriptor length: 8
> >>      designator_type: Logical unit group,  code_set: Binary
> >>      associated with the addressed logical unit
> >>      designator header(hex): 01 06 00 04
> >>      designator:
> >>   00     00 00 00 00                                         ....
> >>   <SNIP>
> >>
> >> So AFAICT, the relative target port, target port group, and logical unit
> >> group being returned from target on v4.5-rc1 code looks correct.
> >>
> >> Serguei, can you confirm with 'sg_inq -Hi /dev/sdX' output on your side
> >> with the v3.10 based target..?
> >>
> >> AFAICT the parsing in scsi_vpd_tpg_id() from commit a8aa3978 looks
> >> correct too.
> >>
> >> Hannes, any ideas..?
> >
> > Ping.
> >
> Please try with my latest scsi_dh_alua patchset posted to linux-scsi.
> That should solve the error attaching devices.
> 

Just to confirm, this was not a target side issue, right..?

Also, since Serguei is seeing this on v4.4 we'll still need some hack
for stable, assuming you're entire patchset won't be in 4.4.y code.  ;)

Are you OK with Mike's original patch, or do you have something better
to submit to Greg-KH..?