Bug 33992 - Applying inappropriate ioctl operation on socket should return ENOTTY
Applying inappropriate ioctl operation on socket should return ENOTTY
Status: RESOLVED CODE_FIX
Product: Networking
Classification: Unclassified
Component: Other
All Linux
: P1 normal
Assigned To: Arnaldo Carvalho de Melo
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-04-27 04:02 UTC by Lifeng Sun
Modified: 2014-10-16 22:00 UTC (History)
2 users (show)

See Also:
Kernel Version: all
Tree: Mainline
Regression: No


Attachments
patch (7.16 KB, patch)
2011-04-27 04:04 UTC, Lifeng Sun
Details | Diff

Description Lifeng Sun 2011-04-27 04:02:54 UTC
ioctl() calls against a socket with an inappropriate ioctl operation
are incorrectly returning EINVAL rather than ENOTTY:

  [ENOTTY]
      Inappropriate I/O control operation.

This bug causes that when the standard input/output/error of ncurses
are redirected to a socket, it fails to set a NOTTY flag hence sticks
on line-buffered mode,

/* ncurses-5.9/ncurses/tinfo/lib_ttyflags.c */
[...]
            for (;;) {
                if (SET_TTY(termp->Filedes, buf) != 0) {
                    if (errno == EINTR)
                        continue;
                    if ((errno == ENOTTY) && (SP_PARM != 0)) /* fail */
                        SP_PARM->_notty = TRUE;
                    result = ERR;
                }
                break;
            }
[...]

This bug is not limited to socket, it also occurs in a lot of, maybe some
hundred, other ioctl operations, while in the patch I only fixed about a
dozen of additional ones in pipe, fifo and character device drivers.
Comment 1 Lifeng Sun 2011-04-27 04:04:49 UTC
Created attachment 55612 [details]
patch
Comment 2 Florian Mickler 2011-05-04 06:09:34 UTC
A patch referencing this bug report has been merged in v2.6.39-rc5-274-g609cfda:

commit 41c31f318a5209922d051e293c61e4724daad11c
Author: Lifeng Sun <lifongsun@gmail.com>
Date:   Wed Apr 27 22:04:51 2011 +0000

    networking: inappropriate ioctl operation should return ENOTTY
Comment 3 jprobeau 2014-10-16 22:00:23 UTC
How would I install/ apply this? This patch fixes a problem, nut I am not sure how to apply/ install it?

Note You need to log in before you can comment on or make changes to this bug.