SCST iSCSI - Kernel panic with Win2003 Initiator

[DMo]

I know this should be strictly a Windows / SCST issue, but wondered if anyone else has run into the same problem.

I have an Ubuntu Server 10.04 X64 system with a recompiled 2.6.32.33 kernel for SCST support. I'm using the 0.18-13 VTL build. The VTL is installed, SCST is compiled/installed, and it all works fine...UNLESS I access the VTL targets using a remote Win2003 system. Both 32 and 64 bit Windows 2003 cause a kernel panic (tested with the 2.06 and 2.08 MSFT initiators.) However, if I use a remote Win2008 R2 initiator, the targets work perfectly.

This happens when the 2003 initiator attempts to query the devices. I assume it is due to some missing/bad data that the 2003 initiator sends. The SCST target doesn't appear to handle it gracefully. But the issue seems fixed with the 2008 initiator, which does not cause the panic. I cannot seem to find any discussion threads/info about SCST and Win2003 initiator compatibility.

To recap, I have tried the following 2.6.32 kernels in conjunction with the last ~3-4 checked-in builds of SCST:
.28
.29
.32
.33

Has anyone run into this issue?

I was able to save the trace from one of the panics, but (for me) it only confirms an issue in one of the SCST modules:


Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.764572] Modules linked in: iscsi_scst libcrc32c crc32c scst_tape scst_changer scst_vdisk scst_disk scst ch osst st mhvtl ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ppdev psmouse parport_pc intel_agp lp serio_raw parport shpchp agpgart i2c_piix4 reiserfs mptspi floppy mptscsih pcnet32 mptbase mii
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.765275]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.765375] Pid: 1306, comm: 3:1:1:00_0 Not tainted (2.6.32.29-DM-iSCSI #1) VMware Virtual Platform
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.765473] EIP: 0060:[<c02170cf>] EFLAGS: 00010286 CPU: 1
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.765539] EIP is at path_init+0x2f/0x1b0
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.765600] EAX: ffffff9c EBX: efb83d58 ECX: 00000000 EDX: 00000000
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.765666] ESI: 00000006 EDI: 00000000 EBP: efb83d1c ESP: efb83d0c
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.765722]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.765933]  c1b04980 efb83d58 00000006 00000000 efb83d34 c0219611 efb83d58 ef511e00
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.766089] <0> 00000006 efb83d58 efb83d40 c02196dc efb83d58 efb83dbc f8fa7fff efb83d80
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.766223] <0> f8fa48f5 efb83d80 ffffffff efb83d80 ef81ce40 00000034 00000034 ef8693c4
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.766563]  [<c0219611>] ? do_path_lookup+0x21/0x90
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.766629]  [<c02196dc>] ? path_lookup+0x1c/0x20
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.766931]  [<f8fa7fff>] ? scst_pr_remove_device_files+0x6f/0x1b0 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.766998]  [<f8fa48f5>] ? tid_equal+0x175/0x350 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.767071]  [<f8fa4f36>] ? scst_pr_add_registrant+0x1c6/0x4d0 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.767147]  [<f8fa4b52>] ? scst_pr_find_reg+0x82/0x120 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.767247]  [<f8fa81e5>] ? scst_pr_sync_device_file+0xa5/0x630 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.767313]  [<f8fa61c4>] ? __scst_pr_register+0xb4/0x200 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.769179]  [<f8fa64c0>] ? scst_pr_register_and_ignore+0x1b0/0x300 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.769244]  [<f8f889b9>] ? scst_get_buf_first+0x39/0x50 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.769314]  [<f8f79d7a>] ? scst_persistent_reserve_out_local+0x3aa/0x550 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.769627]  [<c01443dd>] ? pull_task+0x3d/0x50
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.772516]  [<f8f7a94f>] ? scst_do_local_exec+0x1bf/0x240 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.772576]  [<f8f6f72d>] ? __scst_cmd_get+0x1d/0x60 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.772633]  [<f8f7af7a>] ? scst_exec+0xaa/0x220 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.772722]  [<f92d9da5>] ? scst_cmd_atomic+0x45/0x80 [iscsi_scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.772776]  [<c0143cc3>] ? finish_task_switch+0x43/0xc0
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.772832]  [<f8f7b262>] ? scst_send_for_exec+0x172/0x2c0 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.772892]  [<f8f76ab6>] ? scst_tgt_pre_exec+0x76/0x250 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.772963]  [<f8f7ba5b>] ? scst_process_active_cmd+0x55b/0x5e0 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.773025]  [<f8f7bb67>] ? scst_do_job_active+0x87/0x150 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.773084]  [<f8f7be29>] ? scst_cmd_thread+0x129/0x350 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.773153]  [<c014b4c0>] ? default_wake_function+0x0/0x20
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.773222]  [<f8f7bd00>] ? scst_cmd_thread+0x0/0x350 [scst]
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.773291]  [<c016dad4>] ? kthread+0x74/0x80
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.773338]  [<c016da60>] ? kthread+0x0/0x80
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.773408]  [<c010a2a7>] ? kernel_thread_helper+0x7/0x10
Mar  3 09:42:50 vtl-iscsi kernel: [ 7849.774418] ---[ end trace d0a68f1290a8a182 ]---

[Mark Harvey]

Sorry, I've never worked with scst. However some previous postings with scst logs have shown each scsi command as it is being processed.

Perhaps some kind person this forum can assist on how to enable verbose logging. Once we find the problem OP code, I can check to make sure the vtl is doing the right thing.

Cheers Mark

[Mark Harvey]

Looking at the names of the scst functions, I'd guess that it's the Persistent Reservation thats bring it unstuck.

How you would enable/disable SPR in Win2k3/2k8 is something I don't know.

[gboule]

Hi,

I have the same error with the same environment : W2003 initiator and Ubuntu + mhvtl 0.18.15 + scst

regards

[nia]

Which scst svn branch are you using ? I am having good success with scst svn branch 3177 + mhvtl for both FC and iSCSI even with Microsoft Windows 2003 iSCSI Initiator. I know I have tried other svn branches like 3394 which caused many crashes for my Target System. I am currently running the following:

Oracle Linux 5.6
Using Generic Kernel linux-2.6.33.7
SCST SVN Branch 3177
Compiled for both iscsi & fc
MHVTL Version: 0.18.15-git-c673ddb

nia

[gboule]

Problem is solved : I changed IBM drive by HP LTO ultrium drive and no pb.

I think the w2k3 driver for IBM drives is boggus (according with previous post w2k8 driver is OK)

[eric]

I built a new MHVTL system out of a 32bit X86 system, and I am seeing the same ooops reported above. I have tried both SCST 3227 and 3478 and both MHVTL 0.18.13 and 0.18-16. I have tried both 2.6.32.27 and 2.6.32.41 kernels.  I hit the problem when accessing the library and tape drives, via iSCSI or via FC.  I am also exporting some vdisks, and as long as I do not access the MHVTL library or tape drives, I have no ooops issues.

On an X86_64bit system I have  MHVTL 0.18.13 with SCST 3227 using the 2.6.32.27 kernel running successfully since January.

For others who have seen this problem, has it been on 32bit or 64bit systems?

thanks

eric

[nia]

nia wrote

I am having good success with scst svn branch 3177 + mhvtl for both FC and iSCSI even with Microsoft Windows 2003 iSCSI Initiator. I am currently running the following:

Oracle Linux 5.6
Using Generic Kernel linux-2.6.33.7
SCST SVN Branch 3177
Compiled for both iscsi & fc
MHVTL Version: 0.18.15-git-c673ddb

I am using a 32-bit system and do not have this issue with above configuration.

[Mark Harvey]

A thought would be to use 'wireshare' and capture the traffic between the two hosts.

Then we would have the SCSI op code causing all the problems.

I'm assuming it will be a 'reservation' op code that triggers the error.

Sent from my iPad

On May 31, 2011, at 23:02, "nia [via MHVTL - Linux Virtual Tape Library - Community Forums]"<[hidden email]> wrote:

nia wrote:

I am having good success with scst svn branch 3177 + mhvtl for both FC and iSCSI even with Microsoft Windows 2003 iSCSI Initiator. I am currently running the following:

Oracle Linux 5.6
Using Generic Kernel linux-2.6.33.7
SCST SVN Branch 3177
Compiled for both iscsi & fc
MHVTL Version: 0.18.15-git-c673ddb

I am using a 32-bit system and do not have this issue with above configuration.

---

If you reply to this email, your message will be added to the discussion below:

http://mhvtl-linux-virtual-tape-library-community-forums.966029.n3.nabble.com/SCST-iSCSI-Kernel-panic-with-Win2003-Initiator-tp2695292p3005798.html

To unsubscribe from SCST iSCSI - Kernel panic with Win2003 Initiator, click here.

[eric]

Thanks for the replies.  Wireshark is a great idea.

I am planning on retesting on a 64Bit system once I can free it up.

Eric