Wednesday, June 20, 2007

iSCSI on RHEL4 with openfiler issue

Yesterday, felt good for successfully configured 2-node RAC on VMware 6 workstation. It took my 3 days to troubleshoot all the issues. Then, I think that using iscsi might be easier on configuration and more reliable. So after taking a look on someone's blogs regarding to iscsi. And noticed iscsi is a good stuff, so wanted to give a shot today.

Today, tried to install iscsi on RHEL AS4 with openfiler ( a freeware of NAS). But always could not attach the disk configured in openfiler from initiator.

installed iscsi-initiator-utils-4.0.3.0-4.i386.rpm, sysfsutils-1.2.0-1.i386.rpm

up2date iscsi-initiator-utils


On the target server:
under /opt/openfiler/etc/iscsi/, the xml file on the openfiler server is control the network privilege.
/opt/openfiler/etc/networks.xml file can delete the network configured . i could not find the place under openfiler to delete the network.
and also /etc/initiators.allow


Jun 20 19:36:37 RH4 iscsi: iscsi config check succeeded
Jun 20 19:36:37 RH4 kernel: iscsi-sfnet: Loading iscsi_sfnet version 4:0.1.11-3
Jun 20 19:36:37 RH4 kernel: iscsi-sfnet: Control device major number 254
Jun 20 19:36:37 RH4 iscsi: Loading iscsi driver: succeeded
Jun 20 19:36:43 RH4 iscsid[13333]: version 4:0.1.11-4 variant (15-Jan-2007)
Jun 20 19:36:43 RH4 iscsi: iscsid startup succeeded
Jun 20 19:36:43 RH4 iscsid[13338]: Connected to Discovery Address 10.1.72.125

but could not attach the disk.


If the network is not configured to be allowed access. the following error message will be showed up in initiator /var/log/messages file.

un 20 19:44:47 RH4 iscsid[14750]: cannot make connection to 10.1.72.125:3260: Connection refused
Jun 20 19:44:47 RH4 iscsid[14750]: Connection to Discovery Address 10.1.72.125 failed
Jun 20 19:44:52 RH4 iscsi: iscsid shutdown succeeded
Jun 20 19:44:52 RH4 iscsi: removing driver succeeded
Jun 20 19:44:55 RH4 iscsi: iscsi config check succeeded
Jun 20 19:44:55 RH4 kernel: iscsi-sfnet: Loading iscsi_sfnet version 4:0.1.11-3
Jun 20 19:44:55 RH4 kernel: iscsi-sfnet: Control device major number 254
Jun 20 19:44:55 RH4 iscsi: Loading iscsi driver: succeeded
Jun 20 19:45:01 RH4 iscsid[14880]: version 4:0.1.11-4 variant (15-Jan-2007)
Jun 20 19:45:01 RH4 iscsi: iscsid startup succeeded
Jun 20 19:45:01 RH4 iscsid[14884]: cannot make connection to 10.1.72.125:3260: Connection refused
Jun 20 19:45:01 RH4 iscsid[14884]: Connection to Discovery Address 10.1.72.125 failed
Jun 20 19:45:02 RH4 iscsid[14884]: cannot make connection to 10.1.72.125:3260: Connection refused
Jun 20 19:45:02 RH4 iscsid[14884]: Connection to Discovery Address 10.1.72.125 failed


if only configure target IP:

you get the following message, but can not discover the disk.

Jun 20 19:46:28 RH4 iscsi: iscsi config check succeeded
Jun 20 19:46:28 RH4 kernel: iscsi-sfnet: Loading iscsi_sfnet version 4:0.1.11-3
Jun 20 19:46:28 RH4 kernel: iscsi-sfnet: Control device major number 254
Jun 20 19:46:28 RH4 iscsi: Loading iscsi driver: succeeded
Jun 20 19:46:33 RH4 iscsid[15021]: version 4:0.1.11-4 variant (15-Jan-2007)
Jun 20 19:46:34 RH4 iscsi: iscsid startup succeeded
Jun 20 19:46:34 RH4 iscsid[15025]: Connected to Discovery Address 10.1.72.125

Even configured to allow the initialter's IP access, still can not find the disk.

After configure to allow the whole subnet to access the volume. the problem is solved!!

for example, configure 10.1.72.122/24 : failed. but 10.1.72.0/24 works!.

Then even removed 10.1.72.0/24 from networks.xml file. it still work. I believe that is a bug on openfiler.


command to restart target :

[root@localhost ~]# service iscsi-target restart
Stopping iSCSI target service: [ OK ]
Starting iSCSI target service: [ OK ]
[root@localhost ~]#

command to restart initiator on RHEL:
[root@RH4 etc]# service iscsi restart
Searching for iscsi-based multipath maps
Found 0 maps
Stopping iscsid: [ OK ]
Removing iscsi driver: [ OK ]
Checking iscsi config: [ OK ]
Loading iscsi driver: [ OK ]
mknod: `/dev/iscsictl': File exists
Starting iscsid: [ OK ]



After attached the disk, use the following command to list the iscsi device.

[root@RH4 etc]# iscsi-ls
*******************************************************************************
SFNet iSCSI Driver Version ...4:0.1.11-4(15-Jan-2007)
*******************************************************************************
TARGET NAME : iqn.2006-01.com.openfiler:vol1.crs
TARGET ALIAS :
HOST ID : 10
BUS ID : 0
TARGET ID : 0
TARGET ADDRESS : 10.1.72.125:3260,1
SESSION STATUS : ESTABLISHED AT Wed Jun 20 19:55:19 EDT 2007
SESSION ID : ISID 00023d000001 TSIH 200
*******************************************************************************

or

[root@RH4 etc]# iscsi-device /dev/sdb
/dev/sdb is an iSCSI device


My setting in /etc/iscsi.conf :

HeaderDigest=never
DataDigest=never
ImmediateData=yes
ConnFailTimeout=180
Continuous=no
DiscoveryAddress=10.1.72.125




then use fdisk /dev/sdb , n, p, 1 ... then w to partition the disk.


mke2fs -j -L 

It is extremely important to use disk labels when working with iSCSI storage. Delays in network traffic may cause the LUNs to be discovered in a different order the next time the system is booted. The only way to guarantee that the correct partition will be mounted at the proper mount point is to use disk labels to identify them.


[[root@RH4 mnt]# mke2fs -j -L idisk1 /dev/sdb1
mke2fs 1.35 (28-Feb-2004)
Filesystem label=idisk1
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
122880 inodes, 245527 blocks
12276 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=251658240
8 block groups
32768 blocks per group, 32768 fragments per group
15360 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376

Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 39 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override

mount -L idisk1 /mnt/data1

edit /etc/fstab to put the entry to mount this volume automatically.

LABEL=idisk1 /mnt/data1 ext3 _netdev 0 0



Reference:
1. Redhat Article ID: 7905
2. https://www.openfiler.com/doc/
3. Jeff Hunter's Oracle http://www.idevelopment.info/

2 comments:

Anonymous said...

Thanks for posting your experience. I was struggling with the same issue.

I hope I can over come my issues as well.

Thanks buddy !

Sura

Steve Guan said...

you are welcome. Hope it can help.