I just upgraded from kernel version 3.16.0.4 (Debian Jesse). Now testing on Ubuntu 16.04 4.4.0-22-generic (standard) kernel. Previously NFS server cound handle mounts with vers=3 (or vers=2). But now clients cna only mount with vers=4 (or NFS4 capable clients connect and it negotiates to NFSv4. These are the kernel configs: CONFIG_KERNFS=y CONFIG_NFS_FS=m CONFIG_NFS_V2=m CONFIG_NFS_V3=m CONFIG_NFS_V3_ACL=y CONFIG_NFS_V4=m CONFIG_NFS_SWAP=y CONFIG_NFS_V4_1=y CONFIG_NFS_V4_2=y CONFIG_PNFS_FILE_LAYOUT=m CONFIG_PNFS_BLOCK=m CONFIG_PNFS_OBJLAYOUT=m CONFIG_PNFS_FLEXFILE_LAYOUT=m CONFIG_NFS_V4_1_IMPLEMENTATION_ID_DOMAIN="kernel.org" CONFIG_NFS_V4_1_MIGRATION=y CONFIG_NFS_V4_SECURITY_LABEL=y CONFIG_NFS_FSCACHE=y # CONFIG_NFS_USE_LEGACY_DNS is not set CONFIG_NFS_USE_KERNEL_DNS=y CONFIG_NFS_DEBUG=y CONFIG_NFSD=m CONFIG_NFSD_V2_ACL=y CONFIG_NFSD_V3=y CONFIG_NFSD_V3_ACL=y CONFIG_NFSD_V4=y CONFIG_NFSD_PNFS=y CONFIG_NFSD_V4_SECURITY_LABEL=y # CONFIG_NFSD_FAULT_INJECTION is not set CONFIG_NFS_ACL_SUPPORT=m CONFIG_NFS_COMMON=y CONFIG_NCPFS_NFS_NS=y I have these kernel modules: ./fs/nfsd ./fs/nfsd/nfsd.ko ./fs/nfs_common ./fs/nfs_common/nfs_acl.ko ./fs/nfs ./fs/nfs/filelayout/nfs_layout_nfsv41_files.ko ./fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko ./fs/nfs/nfsv4.ko ./fs/nfs/nfs.ko ./fs/nfs/nfsv2.ko ./fs/nfs/nfsv3.ko I modprobe'd NFSv3. Now it shows up in lsmod # lsmod | grep nfs nfsv3 40960 0 nfs 253952 1 nfsv3 fscache 61440 1 nfs nfsd 319488 13 auth_rpcgss 61440 1 nfsd nfs_acl 16384 2 nfsd,nfsv3 lockd 94208 3 nfs,nfsd,nfsv3 grace 16384 2 nfsd,lockd sunrpc 335872 20 nfs,nfsd,auth_rpcgss,lockd,nfsv3,nfs_acl But even if I restart NFS-kernel-server I get this: # cat /proc/filesystems | grep nfs nodev nfsd nodev nfs nodev nfs4 And of course the problem described above. This is a blocking problem, because OpenVZ containers can on act as NFSv3 clients.
I've kept 99% of the configuration files the same from the Debian to the Ubuntu system, so I'm 99% sure it's a regression in the Kernel. The OpenVZ host VM and it's containers have not changed at all. The OpenVZ Host VM can mount any of the shares with vers=4, but not vers=3. The OpenVZ clients can run rpcinfo -p nfsServerIP and they receive a full listing of all the ports etc.
I've spent all morning testing the 4.6 kernel with NFSv3. This looks like user error to me.
Thanks for responding, but I must object or request further information. Firstly I filed the bug with the 4.4.0-22-generic x86_64 kernel, so how can you mark this resolved saying you've tested with the 4.6 kernel? You've not tested with the kernel where the issue is occurring, so you've not tested at all? Obviously it's unreproducible if you don't test it. ----------------- How can this be user error? I performed this right now one command after another without changing anything else from a VM running a 2.6.32-openvz-042stab113.11-amd64 trying to mount NFS shares, shared by the bare-metal 4.4.0-22-generic x86_64 NFS server. root@openvz /tmp # mount 192.168.122.1:/testnfs/web /mnt/www -o ro,vers=3 mount.nfs: Connection timed out root@openvz /tmp # mount 192.168.122.1:/testnfs/web /mnt/www -o ro,vers=4 root@openvz /tmp # The openvz containers do not have NFSv4. So as a workaround, I've mounted this from the openvz host, then bind-mounted /mnt/www into the container.
I have just performed more testing, on completely separate hardware, with completely different kernel versions for server and client, and I can confirm the exact same NFSv3 bug exists there. This is a completely new system, the following details are completely independent and unrelated to anything I've written above, there is no openvz involved here. All kernel versions and OS's are modern or within the distribution's support lifetime. * NFS Server: Ubuntu 14.04 with self-compiled backport 4.4.8 kernel. The only config I changed on this kernel is I enabled overlayFS and some new Cgroups features to test docker. I did not touch any NFS settings whatsoever. NFS Client: KVM VM Debian8 3.16.0-4-amd64 kernel. I did the exact same test as my previous msg and got the exact same results. I would copy and paste but my VM is in VNC mode and clipboard sharing is not working with it in virt-manager right now. You can see above, it's clear as day. So now you can see I've got 2 completely separate systems, all different kernel versions. Same bug exists in both. So the bug seems to exist in 4.4.0-22 and 4.4.8 kernel. NFSv4 clients can mount, but NFSv3 client cannot.
Yet another test. This one is my RaspberryPi mounting music on my 4.4.8 kernel described in the previous msg. root@raspberrypi:/tmp# mount 192.168.150.180:/media/bob/external/music /tmp/test -o ro,vers=3 mount.nfs: rpc.statd is not running but is required for remote locking. mount.nfs: Either use '-o nolock' to keep locks local, or start statd. mount.nfs: an incorrect mount option was specified root@raspberrypi:/tmp# mount 192.168.150.180:/media/bob/external/music /tmp/test -o ro,vers=3,nolock mount.nfs: Connection timed out root@raspberrypi:/tmp# mount 192.168.150.180:/media/bob/external/music /tmp/test -o ro,vers=4,nolock root@raspberrypi:/tmp# ls test Classical Psychedelic root@raspberrypi:/tmp# uname -r && cat /etc/issue 4.1.19+ Raspbian GNU/Linux 8 \n \l
Sorry, seems to user-error :) I think it's my iptables rules.
The cause of the bug is I had not forced one of the NFS ports to a specific value, and somehow my iptables rules were working with the previous kernel with NFSv3 clients, but the new kernel only worked with NFSv4 clients given my incomplete enforcement of NFS port numbers and iptables script which was common on both systems tested.
I won't be making this mistake again, sorry for the inconvenience :)
Hello lopeonline I am interested in how you fixed your problem with your iptables please ? Dio you remember since it was a long time ago ? Thanks !!