1

I use CentOS 7.

I created simple target/initiator configuration and two acls(ACL) for two users.

However, when I mount on both users and write from the first user, write changes don't propagate to the second user.

It looks like the initiator caches all changes on the host and only submits them to target at the end of the session when he logouts.

Second user sees new changes only if first user logouts and second user unmounts and remounts LUN again. I enabled write-thru on backstore and mapped acls to LUN as rw.

I did configuration and tried on various distributions, still the problem is the same. I could attach configuration as necessary, I think it's pretty general and simple so I don't provide it first hand.

How to allow write changes to propagate to other users that are attached to the same LUN? Thanks.

It seems that it should be expected behaviour but definitive point would be appreciated.

Bulat M.
  • 287
  • 1
  • 5
  • 18
  • Considering that nearly all filesystems' caching is built around the assumption that they're the sole user of the underlying disk, I'll guess this just isn't going to possibly work. – u1686_grawity Jan 20 '17 at 08:09
  • @grawity, but what about NFS, for example? Single export could accessed by multiple users simultaneously where one user's changes propogate to other users immediately. In other case there would be no issues with locking in NFSv3. There should be either definitive description of iSCS behaviour or I misconfigured something. – Bulat M. Jan 20 '17 at 13:50
  • NFS is generally file-oriented (NFSv3 always is), in other words it's a filesystem *on its own*, and thus has clear caching semantics for files. For example, changes are always committed on file close, and such. iSCSI meanwhile is block-oriented – the real filesystem runs entirely on client on top of the iSCSI-provided device, and usually isn't even _aware_ that its backing device is networked (much less that it's shared). So it has no way of propagating notices about changes. – u1686_grawity Jan 20 '17 at 21:27
  • (this can be compared to phones which used the block-oriented Mass Storage for USB file transfer with those which use file-oriented MTP) – u1686_grawity Jan 20 '17 at 21:29
  • Thus behaviour I got is expected? Some reference would be highly appreciated. – Bulat M. Jan 21 '17 at 04:54

1 Answers1

2

iSCSI doesn't do what you think it does. Most file systems are designed to be mounted by only one operating system, but you want two clients to mount the same file system in your iSCSI LUN.

Why This Doesn't Work

Imagine one hard drive that you somehow connected to two computers (or a host and its virtual machine sharing a virtual disk). This is your two iSCSI initiators (2 computers) connecting to your single LUN (1 hard drive).

When you mount the file system on the hard drive to somewhere on both computers, each computer does not have any idea that the other is also using the file system. This can lead to serious data corruption because each computer is stepping over the other's toes.

What You Can Do Instead

Don't let your initiators directly connect to the same LUN over iSCSI.

You need the server to keep track of the changes made to the shared file system. Since the clients don't know about each other's changes, a server must be employed to do that.

Two implementations of this are NFS and CIFS, which are their own file systems.

NFS or CIFS sits as a layer between the client and server's local file system so that the clients don't conflict with each other.

Resources

Basic Walk-Throughs

Above and Beyond

You can set up scalable distributed file systems fairly easily with:

Deltik
  • 19,353
  • 17
  • 73
  • 114
  • What about enterprise environments? Do each client access the same LUN in non-overlaping session with other clients' sessions? I understand your answer, but above situation I described seems a bit odd to me. – Bulat M. Jan 27 '17 at 18:10
  • 1
    @BulatM.: In enterprise environments, each iSCSI LUN is typically assigned to one client. But you want two clients to share one common file system at the same time. This means iSCSI is not the solution you're looking for; rather, you should mount a distributed file system like NFS or CIFS. [(More details on Wikipedia)](https://en.wikipedia.org/wiki/ISCSI#Logical_unit_number) – Deltik Jan 27 '17 at 22:07