Bug 13323

Summary:	2.6.30-rc deadline scheduler performance regression for iozone over NFS
Product:	File System	Reporter:	Rafael J. Wysocki (rjw)
Component:	NFS	Assignee:	bfields
Status:	CLOSED CODE_FIX
Severity:	normal	CC:	trondmy
Priority:	P1
Hardware:	All
OS:	Linux
Kernel Version:	2.6.30-rc	Subsystem:
Regression:	Yes	Bisected commit-id:
Bug Depends on:
Bug Blocks:	13070
Attachments:	SUNRPC: Fix the TCP server's send buffer accounting SUNRPC: Fix the TCP write space reservations for deferred requests SUNRPC: Fix svc_tcp_recvfrom()

Description Rafael J. Wysocki 2009-05-16 21:18:14 UTC

Subject    : 2.6.30-rc deadline scheduler performance regression for iozone over NFS
Submitter  : Jeff Moyer <jmoyer@redhat.com>
Date       : 2009-04-23 14:01
References : http://marc.info/?l=linux-kernel&m=124049547915450&w=4
Notify-Also : Jens Axboe <jens.axboe@oracle.com>
Notify-Also : Olga Kornievskaia <aglo@citi.umich.edu>
Notify-Also : Jim Rees <rees@umich.edu>
Notify-Also : Trond Myklebust <trond.myklebust@fys.uio.no>
Notify-Also : "J. Bruce Fields" <bfields@fieldses.org>

This entry is being used for tracking a regression from 2.6.29.  Please don't
close it until the problem is fixed in the mainline.

Comment 1 Jens Axboe 2009-05-18 08:22:48 UTC

Reassign to Trond, this is a nfs regression, not block layer.

Comment 2 Trond Myklebust 2009-05-19 20:22:24 UTC

Reassigning to Bruce, since this is a server regression rather than a client issue :-)

Comment 3 Trond Myklebust 2009-05-19 20:28:24 UTC

Created attachment 21433 [details]
SUNRPC: Fix the TCP server's send buffer accounting

Currently, the sunrpc server is refusing to allow us to process new RPC
calls if the TCP send buffer is 2/3 full, even if we do actually have
enough free space to guarantee that we can send another request.
The following patch fixes svc_tcp_has_wspace() so that we only stop
processing requests if we know that the socket buffer cannot possibly fit
another reply.

It also fixes the tcp write_space() callback so that we only clear the
SOCK_NOSPACE flag when the TCP send buffer is less than 2/3 full.
This should ensure that the send window will grow as per the standard TCP
socket code.

Comment 4 Trond Myklebust 2009-05-19 20:29:01 UTC

Created attachment 21434 [details]
SUNRPC: Fix the TCP write space reservations for deferred requests

Ensure that deferred requests are accounted for correctly by the write
space reservation mechanism. In order to avoid double counting, remove the
reservation when we defer the request, and save any calculated value, so
that we can restore it when the request is requeued.

Comment 5 Trond Myklebust 2009-05-19 20:29:37 UTC

Created attachment 21435 [details]
SUNRPC: Fix svc_tcp_recvfrom()

Ensure that we immediately read and buffer data from the incoming TCP
stream so that we grow the receive window quickly, and don't deadlock on
large READ or WRITE requests.

Comment 6 Trond Myklebust 2009-05-29 21:20:42 UTC

The offending commit has been reverted in 2.6.30-rc...

Closing bug