Bug discovered in the thread at http://lists.gnu.org/archive/html/bug-coreutils/2016-12/msg00008.html partially quoted below (with the permission of the persons involved).


=== Message 1 ===

--- Bug ---
In a FAT32 file system, if one runs md5sum on a 4,294,967,294-byte file (one byte less than the maximum file size) it succeeds, but if one runs md5sum on a 4,294,967,295-byte file (the maximum file size) it fails with error message "Invalid argument".

--- How to reproduce the bug ---
Create a FAT32 file system in a file "tmp.fs":
   truncate -s 9G tmp.fs
   mkfs.vfat -F 32 tmp.fs
Mount at "/tmp/mounted_tmp/" the file system in file "tmp.fs":
   sudo mkdir /tmp/mounted_tmp/
   sudo mount -o loop,rw,uid=1000,gid=1000 tmp.fs /tmp/mounted_tmp/
Create two files in "/tmp/mounted_tmp/", file "file_1" with 4,294,967,294 bytes and file "file_2" with 4,294,967,295 bytes:
   cd /tmp/mounted_tmp/
   truncate -s 4294967294 file_1
   truncate -s 4294967295 file_2
Run md5sum on the two files "file_1" and "file_2":
   md5sum file_1
   md5sum file_2
The outputs should be respectively (notice that the second output is an error message):
   541249e3205af07b4a03f891185f64a0  file_1
   md5sum: file_2: Invalid argument
Unmount the file system at "/tmp/mounted_tmp/":
   cd ..
   sudo umount /tmp/mounted_tmp/
   sudo rmdir /tmp/mounted_tmp/
Remove the file "tmp.fs".

--- Notes ---
Tested with md5sum 8.25 running on an updated Ubuntu 16.10 with kernel 4.8.0-30-generic.
The same bug affects sha1sum, sha224sum, sha256sum, sha384sum, and sha512sum, but not crc32.


=== Message 2 ===

...
> --- How to reproduce the bug ---
...

I can't repro this with any md5sum version on 4.2.5-300.fc23.x86_64
So I'm guessing a kernel regression.
Can you strace -o /tmp/md5sum.strace md5sum file_2,
and look towards the end of the strace file to identify the syscall returning EINVAL?
In any case I'd direct the issue towards the kernel folks.
...


=== Message 3 ===

> Can you strace -o /tmp/md5sum.strace md5sum file_2,
> and look towards the end of the strace file to identify the syscall
> returning EINVAL?

It is the seventh and eighth lines below:

      ...
   read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) = 32768
   read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) = 32768
   read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) = 32768
   read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) = 32768
   read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 32768) = 32767
   read(3, 0x25c12b0, 8192)                = -1 EINVAL (Invalid argument)
   read(3, 0x25c12b0, 8192)                = -1 EINVAL (Invalid argument)
   write(2, "md5sum: ", 8)                 = 8
   write(2, "file_2", 6)                   = 6
   open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 4
   fstat(4, {st_mode=S_IFREG|0644, st_size=2995, ...}) = 0
   read(4, "# Locale name alias data base.\n#"..., 4096) = 2995
   read(4, "", 4096)                       = 0
   close(4)                                = 0
   open("/usr/share/locale/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
   open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
   open("/usr/share/locale-langpack/en_GB/LC_MESSAGES/libc.mo", O_RDONLY) = 4
   fstat(4, {st_mode=S_IFREG|0644, st_size=3537, ...}) = 0
   mmap(NULL, 3537, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7f53d8d0a000
   close(4)                                = 0
   open("/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
   write(2, ": Invalid argument", 18)      = 18
   write(2, "\n", 1)                       = 1
   lseek(3, 0, SEEK_CUR)                   = 4294967295
   close(3)                                = 0
   close(1)                                = 0
   close(2)                                = 0
   exit_group(1)                           = ?
   +++ exited with 1 +++


=== Message 4 ===

...
>    read(3,
>    "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
>    32768) = 32767

OK we've read all we can, but to verify md5sum will do:
  fread(buffer + 32767, 1, 1, stream)

Then the stdio stream will issue the underlying read()s
I'm not too sure where there are two reads here,
but they shouldn't be returning EINVAL, but just
returning 0 to indicate EOF.

>    read(3, 0x25c12b0, 8192)                = -1 EINVAL (Invalid argument)
>    read(3, 0x25c12b0, 8192)                = -1 EINVAL (Invalid argument)

So it's a kernel bug as suspected.
...