linux: zfs volume snapshot rename does not generate KOBJ_CHANGE, /dev/zvol/.... link is not updated #14223

JKDingwall · 2022-11-24T13:32:52Z

System information

Type	Version/Name
Distribution Name	Ubuntu
Distribution Version	Focal / 20.04
Kernel Version	5.4.0-133-generic (local build of Ubuntu tag)
Architecture	amd64
OpenZFS Version	zfs-2.1.6-1 / zfs-kmod-2.1.6-1

Describe the problem you're observing

When a volume snapshot is renamed zfs rename pool/volume@snap pool/volume@other the corresponding /dev/zvol/... link is not updated. Running udevadm monitor shows no uevent is generated for the action. If the volume (not snapshot) is renamed then uevents are generated and all links are correct.

Describe how to reproduce the problem

root@ubuntu:~# zfs create -V 20G ztank/james

root@ubuntu:~# zfs snapshot ztank/james@1

root@ubuntu:~# zfs snapshot ztank/james@2

root@ubuntu:~# ls -l /dev/zvol/ztank/james*
lrwxrwxrwx 1 root root 12 Nov 24 13:00 /dev/zvol/ztank/james -> ../../zd2736
lrwxrwxrwx 1 root root 12 Nov 24 13:00 /dev/zvol/ztank/james@1 -> ../../zd2752
lrwxrwxrwx 1 root root 12 Nov 24 13:00 /dev/zvol/ztank/james@2 -> ../../zd2768

root@ubuntu:~# zfs rename ztank/james@1 ztank/james@3

root@ubuntu:~# ls -l /dev/zvol/ztank/james*
lrwxrwxrwx 1 root root 12 Nov 24 13:00 /dev/zvol/ztank/james -> ../../zd2736
lrwxrwxrwx 1 root root 12 Nov 24 13:00 /dev/zvol/ztank/james@1 -> ../../zd2752
lrwxrwxrwx 1 root root 12 Nov 24 13:00 /dev/zvol/ztank/james@2 -> ../../zd2768

The expected result is that the @1 symlink is removed and an @3 one is created.

Include any warning/errors/backtraces from the system logs

No messages are recorded in syslog or kern.log which indicate an error.

The text was updated successfully, but these errors were encountered:

JKDingwall · 2022-11-24T14:18:52Z

I don't know if this is interesting:

root@ubuntu:~# /lib/udev/zvol_id /dev/zd2736
ztank/james

root@ubuntu:~# /lib/udev/zvol_id /dev/zd2768
ztank/james@2

root@ubuntu:~# /lib/udev/zvol_id /dev/zd2752
Unable to open device file: /dev/zd2752

root@ubuntu:~# ls -l /dev/zd2752
brw-rw---- 1 root disk 230, 2752 Nov 24 13:44 /dev/zd2752

root@ubuntu:~# strace /lib/udev/zvol_id /dev/zd2752
execve("/lib/udev/zvol_id", ["/lib/udev/zvol_id", "/dev/zd2752"], 0x7fffc0e04428 /* 19 vars */) = 0
brk(NULL)                               = 0x56094f044000
arch_prctl(0x3001 /* ARCH_??? */, 0x7ffdff3ca380) = -1 EINVAL (Invalid argument)
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=38232, ...}) = 0
mmap(NULL, 38232, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f60921ea000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300A\2\0\0\0\0\0"..., 832) = 832
pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784
pread64(3, "\4\0\0\0\20\0\0\0\5\0\0\0GNU\0\2\0\0\300\4\0\0\0\3\0\0\0\0\0\0\0", 32, 848) = 32
pread64(3, "\4\0\0\0\24\0\0\0\3\0\0\0GNU\0\30x\346\264ur\f|Q\226\236i\253-'o"..., 68, 880) = 68
fstat(3, {st_mode=S_IFREG|0755, st_size=2029592, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f60921e8000
pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784
pread64(3, "\4\0\0\0\20\0\0\0\5\0\0\0GNU\0\2\0\0\300\4\0\0\0\3\0\0\0\0\0\0\0", 32, 848) = 32
pread64(3, "\4\0\0\0\24\0\0\0\3\0\0\0GNU\0\30x\346\264ur\f|Q\226\236i\253-'o"..., 68, 880) = 68
mmap(NULL, 2037344, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f6091ff6000
mmap(0x7f6092018000, 1540096, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x22000) = 0x7f6092018000
mmap(0x7f6092190000, 319488, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x19a000) = 0x7f6092190000
mmap(0x7f60921de000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1e7000) = 0x7f60921de000
mmap(0x7f60921e4000, 13920, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f60921e4000
close(3)                                = 0
arch_prctl(ARCH_SET_FS, 0x7f60921e9580) = 0
mprotect(0x7f60921de000, 16384, PROT_READ) = 0
mprotect(0x56094ecf9000, 4096, PROT_READ) = 0
mprotect(0x7f6092221000, 4096, PROT_READ) = 0
munmap(0x7f60921ea000, 38232)           = 0
stat("/dev/zd2752", {st_mode=S_IFBLK|0660, st_rdev=makedev(0xe6, 0xac0), ...}) = 0
openat(AT_FDCWD, "/dev/zd2752", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, "Unable to open device file: /dev"..., 40Unable to open device file: /dev/zd2752
) = 40
exit_group(1)                           = ?
+++ exited with 1 +++

ryao · 2022-11-29T13:48:57Z

Thanks for the report. I expect to see this fixed before Christmas.

JKDingwall · 2022-12-16T08:51:19Z

If there is patch we would be willing to test it.

ryao · 2022-12-23T02:04:26Z

If there is patch we would be willing to test it.

I had hoped that the group that last touched this infrastructure would write the patch, but they have not. I guess I will need to do it, although I cannot do it immediately.

JKDingwall · 2023-01-24T11:56:54Z

This is a bit of a sticking point for us at the moment and we've not managed to come up with a workaround. Is it a simple issue that someone unfamiliar with the zfs code base could resolve with a bit of guidance?

ryao · 2023-03-13T07:49:55Z

Unfortunately, it has been somewhat difficult finding time to do this. I plan to do it, although if you would like it done faster, I suggest contacting Klara Systems to get it done.

stale · 2024-03-13T10:26:48Z

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.

SpiffyB · 2024-05-02T10:05:43Z

Has there been any progress on this? This is a serious blocker for a project I'm working on currently.

JKDingwall · 2024-10-03T08:05:33Z

I can still reproduce this problem in zfs-2.2.6. I've added a couple of printks and for the snapshot rename I can see that zvol_rename_minors_impl() is entered but there is no subsequent call to zvol_os_rename_minor(). I'll continue trying to follow what is happening...

JKDingwall · 2024-10-03T08:20:45Z

The printk I added to zvol_rename_minor_impl():

        printk(KERN_ERR "ZFS: zvol_rename_minors_impl(%s->%s)...\n", oldname, newname);

If I rename a zvol with snapshots, e.g. zfs rename zpool/james zpool/dingwall:

ZFS: zvol_rename_minors_impl(zpool/james->zpool/dingwall)

If I rename a snapshot , e.g. zfs rename zpool/dingwall@apple zpool/dingwall@orange

ZFS: zvol_rename_minors_impl(apple->orange)...

So I suspect the problem is that zvol_rename_minors_impl() is not called with the full zfs path.

JKDingwall · 2024-10-03T09:30:18Z

a0bd735 removed code which prefixed fsname to the snapshot name before calling zvol_rename_minors(). That goes back to 0.7.0 (!) but as long time zfs users I can't remember when we first observed this problem. I'll see if I can reinstate the code which was removed.

-#ifdef _KERNEL
-       oldname = kmem_asprintf("%s@%s", fsname, oldsnapname);
-       newname = kmem_asprintf("%s@%s", fsname, newsnapname);
-       zvol_rename_minors(oldname, newname);
-       strfree(newname);
-       strfree(oldname);
-#endif

…ot rename `zvol_rename_minors()` needs to be given the full path not just the snapshot name. Use code removed in a0bd735 as a guide to providing the necessary values. Closes openzfs#14223 Signed-off-by: James Dingwall <james@dingwall.me.uk>

…ename After renaming a snapshot with 'snapdev=visible' ensure that the /dev entries are updated to reflect the rename. Signed-off-by: James Dingwall <james@dingwall.me.uk>

…ot rename `zvol_rename_minors()` needs to be given the full path not just the snapshot name. Use code removed in a0bd735 as a guide to providing the necessary values. Closes openzfs#14223 Signed-off-by: James Dingwall <james@dingwall.me.uk>

…ename After renaming a snapshot with 'snapdev=visible' ensure that the /dev entries are updated to reflect the rename. Signed-off-by: James Dingwall <james@dingwall.me.uk>

`zvol_rename_minors()` needs to be given the full path not just the snapshot name. Use code removed in a0bd735 as a guide to providing the necessary values. Add ZTS check for /dev changes after snapshot rename. After renaming a snapshot with 'snapdev=visible' ensure that the /dev entries are updated to reflect the rename. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: James Dingwall <james@dingwall.me.uk> Closes openzfs#14223 Closes openzfs#16600

JKDingwall added the Type: Defect Incorrect behavior (e.g. crash, hang) label Nov 24, 2022

stale bot added the Status: Stale No recent activity for issue label Mar 13, 2024

stale bot removed the Status: Stale No recent activity for issue label May 2, 2024

SpiffyB mentioned this issue Oct 2, 2024

Unable to access snapshot files using /.zfs/snapshot/testsnapshot/ #9461

Open

JKDingwall mentioned this issue Oct 3, 2024

issue #14223: fix generation of kernel uevents for snapshot rename on linux #16600

Merged

13 tasks

behlendorf closed this as completed in #16600 Oct 6, 2024

behlendorf closed this as completed in 0b4dcbe Oct 6, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

linux: zfs volume snapshot rename does not generate KOBJ_CHANGE, /dev/zvol/.... link is not updated #14223

linux: zfs volume snapshot rename does not generate KOBJ_CHANGE, /dev/zvol/.... link is not updated #14223

JKDingwall commented Nov 24, 2022 •

edited

Loading

JKDingwall commented Nov 24, 2022 •

edited

Loading

ryao commented Nov 29, 2022

JKDingwall commented Dec 16, 2022

ryao commented Dec 23, 2022

JKDingwall commented Jan 24, 2023

ryao commented Mar 13, 2023

stale bot commented Mar 13, 2024

SpiffyB commented May 2, 2024

JKDingwall commented Oct 3, 2024

JKDingwall commented Oct 3, 2024 •

edited

Loading

JKDingwall commented Oct 3, 2024

linux: zfs volume snapshot rename does not generate KOBJ_CHANGE, /dev/zvol/.... link is not updated #14223

linux: zfs volume snapshot rename does not generate KOBJ_CHANGE, /dev/zvol/.... link is not updated #14223

Comments

JKDingwall commented Nov 24, 2022 • edited Loading

System information

Describe the problem you're observing

Describe how to reproduce the problem

Include any warning/errors/backtraces from the system logs

JKDingwall commented Nov 24, 2022 • edited Loading

ryao commented Nov 29, 2022

JKDingwall commented Dec 16, 2022

ryao commented Dec 23, 2022

JKDingwall commented Jan 24, 2023

ryao commented Mar 13, 2023

stale bot commented Mar 13, 2024

SpiffyB commented May 2, 2024

JKDingwall commented Oct 3, 2024

JKDingwall commented Oct 3, 2024 • edited Loading

JKDingwall commented Oct 3, 2024

JKDingwall commented Nov 24, 2022 •

edited

Loading

JKDingwall commented Nov 24, 2022 •

edited

Loading

JKDingwall commented Oct 3, 2024 •

edited

Loading