-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathopen-iSCSI_tech.txt
1488 lines (1052 loc) · 52.2 KB
/
open-iSCSI_tech.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
=================================================================
Linux* Open-iSCSI
=================================================================
May 20, 2012
Contents
========
- 1. In This Release
- 2. Introduction
- 3. Installation
- 4. Open-iSCSI daemon
- 5. Open-iSCSI Configuration Utility
- 6. Configuration
- 7. Getting Started
- 8. Advanced Configuration
- 9. iSCSI System Info
1. In This Release
==================
This file describes the Linux* Open-iSCSI Initiator. The software was
tested on AMD Opteron (TM) and Intel Xeon (TM).
The latest development release is available at:
http://www.open-iscsi.org
For questions, comments, contributions send e-mail to:
1.1. Features
- highly optimized and very small-footprint data path;
- persistent configuration database;
- SendTargets discovery;
- CHAP;
- PDU header Digest;
- multiple sessions;
2. Introduction
===============
Open-iSCSI project is a high-performance, transport independent,
multi-platform implementation of RFC3720 iSCSI.
Open-iSCSI is partitioned into user and kernel parts.
The kernel portion of Open-iSCSI is a from-scratch code
licensed under GPL. The kernel part implements iSCSI data path
(that is, iSCSI Read and iSCSI Write), and consists of three
loadable modules: scsi_transport_iscsi.ko, libiscsi.ko and iscsi_tcp.ko.
User space contains the entire control plane: configuration
manager, iSCSI Discovery, Login and Logout processing,
connection-level error processing, Nop-In and Nop-Out handling,
and (in the future:) Text processing, iSNS, SLP, Radius, etc.
The user space Open-iSCSI consists of a daemon process called
iscsid, and a management utility iscsiadm.
3. Installation
===============
As of today, the Open-iSCSI Initiator requires a host running the
Linux operating system with kernel version 2.6.16, or later. 2.6.14 and
2.6.15 are partially supported. Known issues with 2.6.14 - .15 support:
- If the device is using a write back cache, during session logout
the cache sync command will fail.
- iscsiadm's -P 3 option will not print out scsi devices.
- iscsid will not automatically online devices.
You need to enable "Cryptographic API" under "Cryptographic options" in the
kernel config. And you must enable "CRC32c CRC algorithm" even if
you do not use header or data digests. They are the kernel options,
CONFIG_CRYPTO and CONFIG_CRYPTO_CRC32C, respectively.
By default the kernel's iSCSI modules will be used. Running:
make
make install
will install the iSCSI tools iscsiadm and iscsid to /sbin.
For 2.6.14 - 2.6.34 the modules in the kernel dir can built and install
by running:
make kernel
When building those modules the kernel source found at
/lib/modules/`uname -a`/build
will be used to compile the open-iscsi modules. To specify a different
kernel to build against use:
make kernel KSRC=<kernel-src>
or cross-compilation:
make kernel KSRC=<kernel-src> KARCH="ARCH=um"
To compile on SUSE Linux you'll have to use
make kernel KSRC=/usr/src/linux \
KBUILD_OUTPUT=/usr/src/linux-obj/<arch>/<config>
where <config> is the kernel configuration to use (eg. 'smp').
To install the kernel modules that were built run:
make install_kernel
This will copy: iscsi_tcp.ko, libiscsi_tcp.ko, libiscsi.ko and
scsi_transport_iscsi to
/lib/modules/`uname -r`/kernel/drivers/scsi/
overwriting existing iscsi modules.
For Debian, be sure to install the linux-headers package that
corresponds to your kernel in order to compile the kernel modules
('aptitude install linux-headers-`uname -r`'). You may also wish to
run 'make -C kernel/ dpkg_divert' before installing kernel modules if
you run a Debian-provided kernel. This will use dpkg-divert(8) to
move the packaged kernel modules out of the way, and ensure that
future kernel upgrades will not overwrite them.
Also, please be aware that the compatibility patches that enable these
iscsi modules to run on kernels older than 2.6.25 will not update the
ib_iser module; you may get warnings related to mismatched symbols on
this driver, in which case you'll be unable to load ib_iser and
open-iscsi simultaneously.
4. Open-iSCSI daemon
====================
The daemon implements control path of iSCSI protocol, plus some management
facilities. For example, the daemon could be configured to automatically
re-start discovery at startup, based on the contents of persistent
iSCSI database (see next section).
For help, run:
./iscsid --help
Usage: iscsid [OPTION]
-c, --config=[path] Execute in the config file (/etc/iscsi/iscsid.conf).
-f, --foreground run iscsid in the foreground
-d, --debug debuglevel print debugging information
-u, --uid=uid run as uid, default is current user
-g, --gid=gid run as gid, default is current user group
-h, --help display this help and exit
-v, --version display version and exit
5. Open-iSCSI Configuration Utility
===================================
Open-iSCSI persistent configuration is implemented as a DBM database
available on all Linux installations.
The database contains two tables:
- Discovery table (/etc/iscsi/send_targets);
- Node table (/etc/iscsi/nodes).
The regular place for iSCSI database files: /etc/iscsi/nodes
The iscsiadm utility is a command-line tool to manage (update, delete,
insert, query) the persistent database.
The utility presents set of operations that a user can perform
on iSCSI nodes, sessions, connections, and discovery records.
Open-iscsi does not use the term node as defined by the iSCSI RFC,
where a node is a single iSCSI initiator or target. Open-iscsi uses the
term node to refer to a portal on a target, so tools like iscsiadm
require that --targetname and --portal argument be used when in node mode.
For session mode, a session id (sid) is used. The sid of a session can be
found by running iscsiadm -m session -P 1. The session id is not currently
persistent and is partially determined by when the session is setup.
Note that some of the iSCSI Node and iSCSI Discovery operations
do not require iSCSI daemon (iscsid) loaded.
For help, run:
./iscsiadm --help
Usage: iscsiadm [OPTION]
-m, --mode <op> specify operational mode op =
<discovery|discoverydb|fw|iface|host|node>
-m discoverydb --type=[type] --interface=[iface...] --portal=[ip:port] \
--print=[N] \
--op=[op]=[NEW | UPDATE | DELETE | NONPERSISTENT] \ --discover
This command will use the discovery record settings
matching the record with type=type and
portal=ip:port]. If a record does not exist, it will
create a record using the iscsid.conf discovery
settings.
By default, it will then remove records for
portals no longer returned. And,
if a portal is returned by the target, then the
discovery command will create a new record or modify
an existing one with values from iscsi.conf and the
command line.
[op] can be passed in multiple times to this
command, and it will alter the node DB manipulation.
If [op] is passed in and the value is
"new", iscsiadm will add records for portals that do
not yet have records in the db.
If [op] is passed in and the value is
"update", iscsiadm will update node records using
info from iscsi.conf and the command line for portals
that are returned during discovery and have
a record in the db.
If [op] is passed in and the value is "delete",
iscsiadm will delete records for portals that
were not returned during discovery.
If [op] is passed in and the value is
"nonpersistent" iscsiadm will not store
the portals found in the node DB. This
only useful with the --login command.
See the example section for more info.
See below for how to setup iscsi ifaces for
software iscsi or override the system defaults.
Multiple ifaces can be passed in during discovery.
For the above commands "print" is optional. If
used, N can be 0 or 1.
0 = The old flat style of output is used.
1 = The tree style with the inteface info is used.
If print is not used the old flay style is used.
-m discoverydb --interface=[iface...] --type=[type] --portal=[ip:port] \
--print=[N] \
--op=[op]=[NEW | UPDATE | DELETE | NONPERSISTENT] \
--discover --login
This works like the previous discoverydb command
with the --login argument passed in will also
log into the portals that are found.
-m discoverydb --portal=[ip:port] --type=[type] \
--op=[op] [--name=[name] --value=[value]]
Perform specific DB operation [op] for
discovery portal. It could be one of:
[new], [delete], [update] or [show]. In case of
[update], you have to provide [name] and [value]
you wish to update
op=NEW will create a new discovery record
using the iscsid.conf discovery settings. If it
already exists, it will be overwritten using
iscsid.conf discovery settings.
op=DELETE will delete the discovery record
and records for the targets found through
that discovery source.
op=SHOW will display the discovery record
values. The --show arguemnt can be used to
force the CHAP passwords to be displayed.
-m discovery --type=[type] --interface=iscsi_ifacename \
--portal=[ip:port] --login --print=[N] \
--op=[op]=[NEW | UPDATE | DELETE | NONPERSISTENT]
perform [type] discovery for target portal with
ip-address [ip] and port [port].
This command will not use the discovery record
settings. It will use the iscsid.conf discovery
settings and it will overwrite the discovery
record with iscsid.conf discovery settings if it
exists. By default, it will then remove records for
portals no longer returned. And,
if a portal is returned by the target, then the
discovery command will create a new record or modify
an existing one with values from iscsi.conf and the
command line.
[op] can be passed in multiple times to this
command, and it will alter the DB manipulation.
If [op] is passed in and the value is
"new", iscsiadm will add records for portals that do
not yet have records in the db.
If [op] is passed in and the value is
"update", iscsiadm will update node records using
info from iscsi.conf and the command line for portals
that are returned during discovery and have
a record in the db.
If [op] is passed in and the value is "delete",
iscsiadm will delete records for portals that
were not returned during discovery.
If [op] is passed in and the value is
"nonpersistent" iscsiadm will not store
the portals found in the node DB.
See the example section for more info.
See below for how to setup iscsi ifaces for
software iscsi or override the system defaults.
Multiple ifaces can be passed in during discovery.
-m discovery --print=[N] display all discovery records from internal
persistent discovery database.
-m node display all discovered nodes from internal
persistent discovery database
-m node --targetname=[name] --portal=[ip:port] \
--interface=iscsi_ifacename] \
[--login|--logout|--rescan|--stats]
-m node --targetname=[name] --portal=[ip:port]
--interface=[driver,HWaddress] \
--op=[op] [--name=[name] --value=[value]]
-m node --targetname=[name] --portal=[ip:port]
--interface=iscsi_ifacename] \
--print=[level]
perform specific DB operation [op] for specific
interface on host that will connect to portal on
target. targetname, portal and interface are optional.
See below for how to setup iscsi ifaces for
software iscsi or override the system defaults.
op could be one of:
[new], [delete], [update] or [show]. In case of
[update], you have to provide [name] and [value]
you wish to update.
[delete] - Note that if a session is using the
node record, the session will be logged out then
the record will be deleted.
Print level can be 0 to 1.
Rescan will perform a SCSI layer scan of the session
to find new LUNs.
Stats prints the iSCSI stats for the session.
-m node --logoutall=[all|manual|automatic]
Logout "all" the running sessions or just the ones
with a node startup value manual or automatic.
Nodes marked as ONBOOT are skipped.
-m node --loginall=[all|manual|automatic]
Login "all" the running sessions or just the ones
with a node startup value manual or automatic.
Nodes marked as ONBOOT are skipped.
-m session display all active sessions and connections
-m session --sid=[sid] [ --print=level | --rescan | --logout ]
--op=[op] [--name=[name] --value=[value]]
perform operation for specific session with
session id sid. If no sid is given the operation
will be performed on all running sessions if possible.
--logout and --op work like they do in node mode,
but in session mode targetname and portal info is
is not passed in.
Print level can be 0 to 2.
1 = Print basic session info like node we are
connected to and whether we are connected.
2 = Print iscsi params used.
3 = Print SCSI info like LUNs, device state.
If no sid and no operation is given print out the
running sessions.
-m iface --interface=iscsi_ifacename --op=[op] [--name=[name] --value=[value]]
--print=level
perform operation on fiven interface with name
iscsi_ifacename.
See below for examples.
-m iface --interface=iscsi_ifacename -C ping --ip=[ipaddr] --packetsize=[size]
--count=[count] --interval=[interval]
-m host --host=hostno|MAC --print=level -C chap --op=[op] --value=[chap_tbl_idx]
Display information for a specific host. The host
can be passed in by host number or by MAC address.
If a host is not passed in then info
for all hosts is printed.
Print level can be 0 to 4.
1 = Print info for how like its state, MAC, and
netinfo if possible.
2 = Print basic session info for nodes the host
is connected to.
3 = Print iscsi params used.
4 = Print SCSI info like LUNs, device state.
-d, --debug debuglevel print debugging information
-V, --version display version and exit
-h, --help display this help and exit
5.1 iSCSI iface setup
=====================
The next sections describe how to setup iSCSI ifaces so you can bind
a session to a NIC port when using software iscsi (section 5.1.1), and
it describes how to setup ifaces for use with offload cards from Chelsio
and Broadcm (section 5.1.2).
5.1.1 How to setup iSCSI interfaces (iface) for binding
=======================================================
If you wish to allow the network susbsystem to figure out
the best path/NIC to use then you can skip this section. For example
if you have setup your portals and NICs on different subnets then
this the following is not needed for software iscsi.
Warning!!!!!!
This feature is experimental. The interface may change. When reporting
bugs, if you cannot do a "ping -I ethX target_portal", then check your
network settings first. Make sure the rp_filter setting is set to 0 or 2
(see Prep section below for more info). If you cannot ping the portal,
then you will not be able to bind a session to a NIC.
What is a scsi_host and iface for software, hardware and partial
offload iscsi?
Software iscsi, like iscsi_tcp and iser, allocate a scsi_host per session
and does a single connection per session. As a result
/sys/class_scsi_host and /proc/scsi will report a scsi_host for
each connection/session you have logged into. Offload iscsi, like
Chelsio cxgb3i, allocates a scsi_host for each PCI device (each
port on a HBA will show up as a different PCI device so you get
a scsi_host per HBA port).
To manage both types of initiator stacks, iscsiadm uses the interface (iface)
structure. For each HBA port or for software iscsi for each network
device (ethX) or NIC, that you wish to bind sessions to you must create
a iface config /etc/iscsi/ifaces.
Prep:
The iface binding feature requires the sysctl setting
net.ipv4.conf.default.rp_filter to be set to 0 or 2. This can be set
in /etc/sysctl.conf by having the line:
net.ipv4.conf.default.rp_filter = N
where N is 0 or 2. Note that when setting this you may have to reboot
the box for the value to take effect.
rp_filter information from Documentation/networking/ip-sysctl.txt:
rp_filter - INTEGER
0 - No source validation.
1 - Strict mode as defined in RFC3704 Strict Reverse Path
Each incoming packet is tested against the FIB and if the interface
is not the best reverse path the packet check will fail.
By default failed packets are discarded.
2 - Loose mode as defined in RFC3704 Loose Reverse Path
Each incoming packet's source address is also tested against the FIB
and if the source address is not reachable via any interface
the packet check will fail.
Running:
# iscsiadm -m iface
iface0 qla4xxx,00:c0:dd:08:63:e8,20.15.0.7,default,iqn.2005-06.com.redhat:madmax
iface1 qla4xxx,00:c0:dd:08:63:ea,20.15.0.9,default,iqn.2005-06.com.redhat:madmax
Will report iface configurations that are setup in /etc/iscsi/ifaces.
The format is:
iface_name transport_name,hwaddress,ipaddress,net_ifacename,initiatorname
For software iscsi, you can create the iface configs by hand, but it is
reccomended that you use iscsiadm's iface mode. There is a iface.example in
/etc/iscsi/ifaces which can be used as a template for the daring.
For each network object you wish to bind a session to you must create
a seperate iface config in /etc/iscsi/ifaces and each iface config file
must have a unique name which is less than or equal to 64 characters.
Example:
If you have NIC1 with MAC address 00:0F:1F:92:6B:BF and NIC2 with
MAC address 00:C0:DD:08:63:E7 and you wanted to do software iscsi over
TCP/IP. Then in /etc/iscsi/ifaces/iface0 you would enter:
iface.transport_name = tcp
iface.hwaddress = 00:0F:1F:92:6B:BF
and in /etc/iscsi/ifaces/iface1 you would enter:
iface.transport_name = tcp
iface.hwaddress = 00:C0:DD:08:63:E7
Warning: Do not name a iface config file "default" or "iser".
They are special value/file that is used by the iscsi tools for
backward compatibility. If you name a iface default or iser, then
the behavior is not defined.
To use iscsiadm to create iface0 above for you run:
(This will create a new empty iface config. If there was already a iface
with the name "iface0" this command will overwrite it.)
# iscsiadm -m iface -I iface0 --op=new
(This will set the hwaddress.)
# iscsiadm -m iface -I iface0 --op=update -n iface.hwaddress -v 00:0F:1F:92:6B:BF
If you had sessions logged in iscsiadm will not update, overwrite
a iface. You must log out first. If you have a iface bound to a node/portal
but you have not logged in then, iscsiadm will update the config and
all existing bindings.
You should now skip to 5.1.3 to see how to log in using the iface and for
some helpful management commands.
5.1.2 Setting up a iface for a iSCSI offload card
=================================================
This section describes how to setup ifaces for use with Chelsio, Broadcom and
QLogic cards.
By default, iscsiadm will create a iface for each Broadcom, QLogic and Chelsio
port. The iface name will be of the form:
$transport/driver_name.$MAC_ADDRESS
Running:
# iscsiadm -m iface
default tcp,<empty>,<empty>,<empty>,<empty>
iser iser,<empty>,<empty>,<empty>,<empty>
cxgb3i.00:07:43:05:97:07 cxgb3i,00:07:43:05:97:07,<empty>,<empty>,<empty>
qla4xxx.00:0e:1e:04:8b:2e qla4xxx,00:0e:1e:04:8b:2e,<empty>,<empty>,<empty>
Will report iface configurations that are setup in /etc/iscsi/ifaces.
The format is:
iface_name transport_name,hwaddress,ipaddress,net_ifacename,initiatorname
iface_name: name of iface
transport_name: name of driver
hwaddress: MAC address
ipaddress: IP address to use for this port
net_iface_name: Net_ifacename will be <empty> because change between
reboots. It is used for software iSCSI's vlan or alias binding.
initiatorname: Initiatorname to be used if you want to override the
default one in /etc/iscsi/initiatorname.iscsi.
To display these values in a more friendly way, run:
iscsiadm -m iface -I cxgb3i.00:07:43:05:97:07
# BEGIN RECORD 2.0-871
iface.iscsi_ifacename = cxgb3i.00:07:43:05:97:07
iface.net_ifacename = <empty>
iface.ipaddress = <empty>
iface.hwaddress = 00:07:43:05:97:07
iface.transport_name = cxgb3i
iface.initiatorname = <empty>
# END RECORD
Before you can use the iface, you must set the IP address for the port
with the following command:
iscsiadm -m iface -I cxgb3i.00:07:43:05:97:07 -o update -n iface.ipaddress -v 20.15.0.66
Note1.
For the name of the value we want to update we use the name from
the "iscsiadm -m iface -I cxgb3i.00:07:43:05:97:07" command which is
"iface.ipaddress".
Note2.
For QLogic ports after updating the iface record, for network settings to take
effect, one must apply or applyall the settings.
iscsiadm -m iface -I qla4xxx.00:0e:1e:04:8b:2e -o apply or
iscsiadm -m iface -H 00:0e:1e:04:8b:2e -o applyall
With operation "apply" network setting for the specified iface will take effect.
With operation "applyall" network settings for all ifaces on a specific host
will take take effect. The host can be specified using the -H/--host argument
by either the MAC address of the host or the host number.
Here is an example of setting multiple IPv6 address on single iSCSI interface
port.
First interface (no need to set iface_num, it is 0 by default)
iscsiadm -m iface -I qla4xxx.00:0e:1e:04:8b:2a -o update \
-n iface.ipaddress -v fec0:ce00:7014:0041:1111:2222:1e04:9392
Create the second interface if it does not exist
iscsiadm -m iface -I qla4xxx.00:0e:1e:04:8b:2a.1 -op=new
iscsiadm -m iface -I qla4xxx.00:0e:1e:04:8b:2a -o update \
-n iface.iface_num -v 1 (iface_num is mandatory for second iface)
iscsiadm -m iface -I qla4xxx.00:0e:1e:04:8b:2a -o update \
-n iface.ipaddress -v = fec0:ce00:7014:0041:1111:2222:1e04:9393
iscsiadm -m iface -H 00:0e:1e:04:8b:2a --op=applyall
Note: If there are common settings for multiple interfaces then the
settings from 0th iface would be considered valid.
Now, we can use this iface to login into targets, which is described in the
next section.
5.1.3 Discoverying iSCSI targets/portals
========================================
Be aware that iscsiadm will use the default route to do discovery. It will
not use the iface specified. So if you are using a offload card, you will
need a seperate network connection to the target for discovery purposes.
*This will be fixed in the next version of open-iscsi*
For compatibility reasons, when you run iscsiadm to do discovery, it
will check for interfaces in /etc/iscsi/iscsi/ifaces that are using
tcp for the iface.transport and it will bind the portals that are discovered
so that they will be logged in through those ifaces. This behavior can also
be overriden by passing in the interfaces you want to use. For the case
of offload like with cxgb3i and bnx2i this is required because the transport
will not be tcp.
For example if you had defined two interface but only wanted to use one
you can use the --interface/-I argument:
iscsiadm -m discoverydb -t st -p ip:port -I iface1 --discover -P 1
If you had defined interfaces but wanted the old behavior, where
we do not bind a session to a iface, then you can use the special iface
"default":
iscsiadm -m discoverydb -t st -p ip:port -I default --discover -P 1
And if you did not define any interfaces in /etc/iscsi/ifaces and do
not pass anything into iscsiadm, running iscsiadm will do the default
behavior, where we allow the network subsystem to decide which
device to use.
If you later want to remove the bindings for a specific target and
iface then you can run:
iscsiadm -m node -T my_target -I iface0 --op=delete
To do this for a specific portal on a target run:
iscsiadm -m node -T my_target -p ip:port -I iface0 --op=delete
If you wanted to delete all bindinds for iface0 then you can run
iscsiadm -m node -I iface0 --op=delete
And for equalogic targets it is sometimes useful to remove by just portal
iscsiadm -m node -p ip:port -I iface0 --op=delete
To now log into targets it is the same as with sofware iscsi. See section
7 for how to get started.
5.2 iscsiadm examples
=====================
Usage examples using the one-letter options (see iscsiadm man page
for long options):
Discovery mode:
- SendTargets iSCSI Discovery using the default driver and interface and
using the discovery settings for the discovery record with the
ID [192.168.1.1:3260].
./iscsiadm -m discoverydb -t st -p 192.168.1.1:3260 --discover
This will search /etc/iscsi/send_targets for a record with the
ID [portal = 192.168.1.1:3260 and type = sendtargets. If found it
will perform discovery using the settings stored in the record.
If a record does not exist, it will be created using the iscsid.conf
discovery settings.
The argument to -p may also be a hostname instead of an address.
./iscsiadm -m discoverydb -t st -p smoehost --discover
For the ifaces, iscsiadm will first search /etc/iscsi/ifaces for
interfaces using software iscsi. If any are found then nodes found
during discovery will be setup so that they can logged in through
those interfaces. To specify a specific iface, pass the
-I argument for each iface.
- SendTargets iSCSI Discovery updating existing target records:
./iscsiadm -m discoverydb -t sendtargets -p 192.168.1.1:3260 \
-o update --discover
If there is a record for targetX and portalY exists in the DB, and
is returned during discovery, it will be updated with the info
from the iscsi.conf. No new portals will be added and stale
portals will not be removed.
- SendTargets iSCSI Discovery deleting existing target records:
./iscsiadm -m discoverydb -t sendtargets -p 192.168.1.1:3260 \
-o delete --discover
If there a record for targetX and portalY exists in the DB, but
is not returned during discovery it will be removed from the DB.
No new portals will be added and existing portal records will not
be changed.
Note: If a session is logged into portal we are going to delete
a record for, it will be logged out then the record will be
deleted.
- SendTargets iSCSI Discovery adding new records:
./iscsiadm -m discoverydb -t sendtargets -p 192.168.1.1:3260 \
-o new --discover
If there targetX and portalY is returned during discovery and does
not have a record, it will be added. Existing records are not
modified.
- SendTargets iSCSI Discovery using multiple ops:
./iscsiadm -m discoverydb -t sendtargets -p 192.168.1.1:3260 \
-o new -o delete --discover
This command will add new portals and delete records for portals
no longer returned. It will not change the record information for
existing portals.
- SendTargets iSCSI Discovery in nonpersistent mode:
./iscsiadm -m discoverydb -t sendtargets -p 192.168.1.1:3260 \
-o nonpersistent --discover
This command will perform discovery, but not manipulate the node DB.
- SendTargets iSCSI Discovery with a specific interface. If you
wish to only use a subset of the interfaces in /etc/iscsi/ifaces
then you can pass them in during discovery:
./iscsiadm -m discoverydb -t sendtargets -p 192.168.1.1:3260 \
--interface=iface0 --interface=iface1 --discover
Note that for software iscsi, we let the network layer select
which NIC to use for discovery, but for later logins iscsiadm
will use the NIC defined in the iface config.
qla4xxx support is very basic and experimental. It does not store
the record info in the card's FLASH or the node DB, so you must
rerun discovery every time the driver is reloaded.
- Manipulate SendTargets DB.
Create new SendTargets discovery record or overwrite an existing
discovery record with iscsid.conf discovery settings.
./iscsiadm -m discoverydb -t sendtargets -p 192.168.1.1:3260 -o new
See discovery settings.
./iscsiadm -m discoverydb -t sendtargets -p 192.168.1.1:3260 -o show
See hidden discovery settings like CHAP passwords
./iscsiadm -m discoverydb -t sendtargets -p 192.168.1.1:3260 \
-o show --show
Set discovery setting.
./iscsiadm -m discoverydb -t sendtargets -p 192.168.1.1:3260 \
-o update -n name -v value
Delete discovery record. This will also delete the records for
the targets found through the discovery source.
./iscsiadm -m discoverydb -t sendtargets -p 192.168.1.1:3260 -o delete
Node mode. In node mode you can specify which records you want to log
into by specifying the targetname, ip address, port or interface
(if specifying the interface it must already be setup in the node db).
iscsiadm will search the node db, for records which match the values
you pass in, so if you pass in the targetname and interface, iscsiadm
will search for records with those values and operate on only them.
Passing in none of them will result in all node records being operated on.
- iSCSI Login to all portals on every node/starget through each interface
set in the db:
./iscsiadm -m node -l
- iSCSI login to all portals on a node/target through each interface set
in the db:
./iscsiadm -m node -T iqn.2005-03.com.max -l
- iSCSI login to a specific portal through each interface set in the db:
./iscsiadm -m node -T iqn.2005-03.com.max -p 192.168.0.4:3260 -l
To specify a IPv6 address the following can be used:
./iscsiadm -m node -T iqn.2005-03.com.max \
-p 2001:c90::211:9ff:feb8:a9e9 -l
The above command would use the default port, 3260. To specify a
port use the following:
./iscsiadm -m node -T iqn.2005-03.com.max \
-p [2001:c90::211:9ff:feb8:a9e9]:3260 -l
To specify a hostname the following can be used:
./iscsiadm -m node -T iqn.2005-03.com.max -p somehost -l
- iSCSI Login to a specific portal through the NIC setup as iface0:
./iscsiadm -m node -T iqn.2005-03.com.max -p 192.168.0.4:3260 \
-I iface0 -l
- iSCSI Logout to all portals on every node/starget through each interface
set in the db:
./iscsiadm -m node -u
Warning: this does not check startup values like the logout/login all
option. Do not use this if you are running iscsi on your root disk.
- iSCSI logout to all portals on a node/target through each interface set
in the db:
./iscsiadm -m node -T iqn.2005-03.com.max -u
- iSCSI logout to a specific portal through each interface set in the db:
./iscsiadm -m node -T iqn.2005-03.com.max -p 192.168.0.4:3260 -u
- iSCSI Logout to a specific portal through the NIC setup as iface0:
./iscsiadm -m node -T iqn.2005-03.com.max -p 192.168.0.4:3260 \
-I iface0
- Changing iSCSI parameter:
./iscsiadm -m node -T iqn.2005-03.com.max -p 192.168.0.4:3260 \
-o update -n node.cnx[0].iscsi.MaxRecvDataSegmentLength -v 65536
You can also change paramaters for multiple records at once, by
specifying different combinations of the target, portal and interface
like above.
- Adding custom iSCSI portal:
./iscsiadm -m node -o new -T iqn.2005-03.com.max \
-p 192.168.0.1:3260,2 -I iface4
The -I/--interface is optional. If not passed in, "default" is used.
For tcp or iser, this would allow the network layer to decide what is
best.
Note that for this command the target portal group tag (TPGT) should
be passed in. If it is not passed in on the initial creation command
then the user must run iscsiadm again to set the value. Also
if the TPGT is not initially passed in, the old behavior of not
tracking whether the record was statically or dynamically created
is used.
- Adding custom NIC config to multiple targets:
./iscsiadm -m node -o new -I iface4
This command will add a interface config using the iSCSI and SCSI
settings from iscsid.conf to every target that is in the node db.
- Removing iSCSI portal:
./iscsiadm -m node -o delete -T iqn.2005-03.com.max -p 192.168.0.4:3260
You can also delete multiple records at once, by specifying different
combinations of the target, portal and interface like above.
- Display iSCSI portal onfiguration:
./iscsiadm -m node -T iqn.2005-03.com.max -p 192.168.0.4:3260
or
./iscsiadm -m node -o show -T iqn.2005-03.com.max -p 192.168.0.4:3260
You can also display multiple records at once, by specifying different
combinations of the target, portal and interface like above.
Note: running "iscsiadm -m node" will only display the records. It
will not display the configuration info. You must run,
"iscsiadm -m node -o show".
- Show all node records:
./iscsiadm -m node
This will print the nodes using the old flat format where the
interface and driver are not displayed. To display that info
use the -P argument with the arguent "1":
./iscsiadm -m node -P 1
- Show all records in discovery database:
./iscsiadm -m discovery
- Show all records in discovery database and show the targets that
were discovered from each record:
./iscsiadm -m discovery -P 1
- Display session statistics:
./iscsiadm -m session -r 1 --stats
This function also works in node mode. Instead of the "-r $sid"
argument, you would pass in the node info like targetname and/or portal,
and/or interface.
- Perform a SCSI scan on a session
./iscsiadm -m session -r 1 --rescan
This function also works in node mode. Instead of the "-r $sid"
argument, you would pass in the node info like targetname and/or portal,
and/or interface.
Note: Rescanning does not delete old LUNs. It will only pick up new
ones.
- Display running sessions:
./iscsiadm -m session -P 1
6. Configuration
================
The default configuration file is /etc/iscsi/iscsid.conf. This file contains
only configuration that could be overwritten by iSCSI Discovery,
or manualy updated via iscsiadm utility. Its OK if this file does not
exist in which case compiled-in default configuration will take place
for newer discovered Target nodes.
See the man page and the example file for the current syntax.
The manpages for iscsid, iscsiadm are in the doc subdirectory and can be
installed in the appropriate man page directories and need to be manually
copied into e.g. /usr/local/share/man8.
7. Getting Started
==================
There are three steps needed to set up a system to use iSCSI storage:
7.1. iSCSI startup using the init script or manual startup.
7.2. Discover targets.
7.3. Automate target logins for future system reboots.
The init scripts will start the iSCSI daemon and log into any
portals that are set up for automatic login (discussed in 7.2)
or discovered through the discover daemon iscsid.conf params
(discussed in 7.1.2).
If your distro does not have a init script, then you will have to start the
daemon and log into the targets manually.
7.1.1 iSCSI startup using the init script
-----------------------------------------------
Red Hat or Fedora:
-----------------
To start open-iscsi in Red Hat/Fedora you can do:
service open-iscsi start
To get open-iscsi to automatically start at run time you may have to
run:
chkconfig --level <levels> open-iscsi on
Where <levels> are the run levels.