Bug 16077 - Drop is video frame rate in kernel .34 [auto-exposure related]
Summary: Drop is video frame rate in kernel .34 [auto-exposure related]
Status: RESOLVED OBSOLETE
Alias: None
Product: v4l-dvb
Classification: Unclassified
Component: webcam (show other bugs)
Hardware: All Linux
: P1 enhancement
Assignee: webcam
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-05-30 14:29 UTC by Vish
Modified: 2012-07-20 12:42 UTC (History)
4 users (show)

See Also:
Kernel Version: 2.6.34
Subsystem:
Regression: Yes
Bisected commit-id:


Attachments
lspci -vvnn (14.20 KB, text/plain)
2010-05-30 14:29 UTC, Vish
Details
lsusb (370 bytes, text/plain)
2010-05-30 14:30 UTC, Vish
Details
dmesg (56.94 KB, application/octet-stream)
2010-05-30 14:32 UTC, Vish
Details
cheese recording with kernel .33 (672.03 KB, video/ogg)
2010-05-30 14:37 UTC, Vish
Details
cheese recording with kernel .34 (275.68 KB, video/ogg)
2010-05-30 14:39 UTC, Vish
Details
v4l-info .33 (2.32 KB, application/octet-stream)
2010-06-04 18:27 UTC, Vish
Details
v4l-info .35 (3.17 KB, application/octet-stream)
2010-06-04 18:29 UTC, Vish
Details
Option to turn off auto-exposure available (22.07 KB, image/png)
2010-07-02 11:07 UTC, Vish
Details

Description Vish 2010-05-30 14:29:52 UTC
Created attachment 26580 [details]
lspci -vvnn

I had reported a bug in Ubuntu for a different video problem:
https://bugs.launchpad.net/bugs/543485

I was testing the upstream kernels[.33 and .34] for the bug and While testing the upstream kernels, in Kernel .34 I noticed:

- A huge improvement in the picture quality
- But a drastic reduction in the video frame rates.

While the picture quality improvement is a welcome improvement , this drop in frame rate makes recording the videos a bit difficult

This is a regression from the older kernels and I'm noticing this only on kernel .34

~$ uname -a
Linux vish-laptop 2.6.34-999-generic #201005121008 SMP Wed May 12 10:11:28 UTC 2010 i686 GNU/Linux
Comment 1 Vish 2010-05-30 14:30:58 UTC
Created attachment 26581 [details]
lsusb
Comment 2 Vish 2010-05-30 14:32:17 UTC
Created attachment 26582 [details]
dmesg
Comment 3 Vish 2010-05-30 14:37:05 UTC
Created attachment 26583 [details]
cheese recording with kernel .33

Attaching a recording of the video with kernel .33.

The video frame rate is better here.
Comment 4 Vish 2010-05-30 14:39:04 UTC
Created attachment 26584 [details]
cheese recording with kernel .34

The video frame rate is low in kernel .34 , while the picture quality hugely improved.
Comment 5 Mauro Carvalho Chehab 2010-06-03 01:47:02 UTC
commit acef4a407ed6e0a9ed87a2747be592fe49e64bdd added the proper sensor support for this board. From the driver, this device supports two different resolutions:
qvga (320x240) and vga (640x480).

Due to sensor's properties, the lowest resolution you have, the highest frame rate it can provide.

Probably, cheese is using different resolutions on kernel 2.6.33 and 2.6.34.

What resolutions and frame rates you're getting?
Comment 6 Hans de Goede 2010-06-03 09:06:37 UTC
I notice in the original bug report that you claim that the lower framerate
clip with 2.6.34 has "much better quality", could you define this a bit better.

I think that what is happening is the code for the new (correct) sensor is
setting a higher exposure value (and thus a lighter / less dark image), but
setting a higher exposure value comes at the cost of framerate. As the framerate
can never be higher then 1 / exposure_time_for_1_frame.

2 things:

1) Go the preferences in cheese, and see which resolutions you can select, and
   make sure you are using the same resolution in 2.6.34 and 2.6.33

2) Start a v4l2 control panel applet, like v4l2ucp or gtk-v4l, and try playing
   around with the controls (note the controls inside cheese are software not
   hardware controls so don't use those).

Regards,

Hans
Comment 7 Vish 2010-06-04 18:27:51 UTC
Created attachment 26651 [details]
v4l-info .33

(In reply to comment #5)
> Due to sensor's properties, the lowest resolution you have, the highest frame
> rate it can provide.
> 
> Probably, cheese is using different resolutions on kernel 2.6.33 and 2.6.34.

The video image is the same as the video preview I get, when I check with gstreamer-properties [video input test]. 
AFAIK , cheese just uses the gstreamer pipeline output.

> 
> What resolutions 

Resolutions, I got (320x240) and (640x480) earlier .
But was only able to use (320x240) with earlier kernels. 

At (640x480) I would get a *lot* of errors:
gspca: ISOC data error: [109] len=1024, status=-71
which was what my downstream bug was about: 
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/543485

This error is however *very* much rarer in kernel .34

> and frame rates you're getting?

How do I check this exactly ? 

The "framerate" I referred to earlier was just based on the /perceived/ skipped frames I can notice in the gstreamer preview and in cheese recording.

(In reply to comment #6)
> I notice in the original bug report that you claim that the lower framerate
> clip with 2.6.34 has "much better quality", could you define this a bit
> better.

Better quality , I meant the less dark image in the kernel .34
With kernel .33 and earlier, increasing the brightness [in cheese] would not even achieve as clear an image as the image with .34
Could be the better exposure.

> 
> 2 things:
> 
> 1) Go the preferences in cheese, and see which resolutions you can select,
> and
>    make sure you are using the same resolution in 2.6.34 and 2.6.33

As mentioned above I only got and get (320x240) and (640x480) resolutions.
I recorded at (320x240) in both kernels , since that resolution works in both kernels.

*But* , the webcam is actually a 1.3Megapixel camera. [it is written 1.3MEGA on the side of the built-in webcam's hole]

I tested the kernel .35 also , Attaching the v4l-info from the two kernels , [.33 and .35]

> 
> 2) Start a v4l2 control panel applet, like v4l2ucp or gtk-v4l, and try
> playing
>    around with the controls (note the controls inside cheese are software not
>    hardware controls so don't use those).

I tried this [Installed the v4l2ucp (2.0.2-2)]. 
In .34/.35 , tried reducing the brightness, saturation, contrast, sharpness, but nothing helped , the frame rate is just stays the same and image is skipped as in the video I posted above. 

These brightness,contrast,saturation options dont exist in earlier kernels[<.33] , which have only two options: 1> sharpness and 2> Light frequency filter which by default is set at 50Hz [options: No / 60 Hz] didnt notice any major changes with changing that setting though.
Comment 8 Vish 2010-06-04 18:29:10 UTC
Created attachment 26652 [details]
v4l-info .35

v4l-info from kernel 35.

Do let me know if I need to provide any other info.
Comment 9 Hans de Goede 2010-06-05 10:57:07 UTC
Thanks for the info. Ok I think this is an exposure "issue" (or rather expected behaviour). You're probably testing indoors, try testing outdoors on a sunny day, then either the image should get completely washed out (if we're using a fixed exposre) and we need to figure out how to control the exposure, or if we are using auto exposure the camera will adjust and we should get a higher framerate with some luck.

There is very little we can do here webcam sensors need to receive light to build up an image, in a low light environment (which indoors usually is), the only way to receive enough light, is to take a lot of time receiving light, this is the exposure time. It is not unheard of to need an exposure time of 200ms, which automatically limits the framerate to 5 fps.
Comment 10 Vish 2010-06-22 10:29:30 UTC
(In reply to comment #9)
> , or if we
> are using auto exposure the camera will adjust and we should get a higher
> framerate with some luck.

WOW! you are right , the camera seems to use auto-exposure.
I tested this at noon and the frame rate is good.

Is there a way we can turn off auto-exposure and control the frame-rate here so that it would work as it did with the earlier kernels? 
[just wondering , how were the old drivers able to maintain the frame rate even in low light]
Comment 11 Vish 2010-06-25 04:25:22 UTC
(In reply to comment #9)
> Thanks for the info. Ok I think this is an exposure "issue" (or rather
> expected
> behaviour). You're probably testing indoors, try testing outdoors on a sunny
> day, then either the image should get completely washed out (if we're using a
> fixed exposre) and we need to figure out how to control the exposure, or if
> we
> are using auto exposure the camera will adjust and we should get a higher
> framerate with some luck.

I tested the older kernels[.32,.33] too on a sunny day , but the image doesnt get washed out. [its maybe also having auto exposure but less sensitive during dark lights though? ] 
So the problem of the low framerate is with the new auto-exposure feature.

Would be great if we could turn it off at times.
Comment 12 Hans de Goede 2010-06-25 06:30:17 UTC
(In reply to comment #11)
> (In reply to comment #9)
> 
> Would be great if we could turn it off at times.

I'm afraid that that is not really doable. The driver is based on usb traces made from running the windows driver. Before we were running the wrong sequence of usb commands (a sequence meant for another sensor). Now we are programming it using the correct sequence. But we don't know what most of the commands in the sequence do so we cannot add little tweaks like the one you are requesting.

I believe this bug can be closed now, but I don't have the rights to do so I'm afraid.
Comment 13 Vish 2010-06-25 06:59:06 UTC
(In reply to comment #12)
> 
> I'm afraid that that is not really doable. The driver is based on usb traces
> made from running the windows driver. Before we were running the wrong
> sequence
> of usb commands (a sequence meant for another sensor). Now we are programming
> it using the correct sequence. But we don't know what most of the commands in
> the sequence do so we cannot add little tweaks like the one you are
> requesting.
> 
> I believe this bug can be closed now, but I don't have the rights to do so
> I'm
> afraid.

Oh .. :(  
I'v set it to an enhancement. Maybe someone could figure it out in future?
Or if the authors dont want to do it they can close the bug. :(
Comment 14 Vish 2010-07-02 11:07:58 UTC
Created attachment 26998 [details]
Option to turn off auto-exposure available

(In reply to comment #12)
> (In reply to comment #11)
> > (In reply to comment #9)
> > 
> > Would be great if we could turn it off at times.
> 
> I'm afraid that that is not really doable. The driver is based on usb traces
> made from running the windows driver. Before we were running the wrong
> sequence
> of usb commands (a sequence meant for another sensor). Now we are programming
> it using the correct sequence. But we don't know what most of the commands in
> the sequence do so we cannot add little tweaks like the one you are
> requesting.

Oh! I was just doing some traces for another regression Bug 16270 and I'v noticed that there is an option to turn this auto-exposure [auto gain] off in Windows.

If I ran a trace with that option turned off , will it help to fix this bug?
Comment 15 Hans de Goede 2010-07-02 12:49:40 UTC
(In reply to comment #14)
> 
> Oh! I was just doing some traces for another regression Bug 16270 and I'v
> noticed that there is an option to turn this auto-exposure [auto gain] off in
> Windows.
> 
> If I ran a trace with that option turned off , will it help to fix this bug?

It might, if you collect traces both with and without the autoexposure option being on and mail them to the driver author / maintainer Jean-François Moine he might be able to add such an option under Linux.
Comment 16 Vish 2010-07-02 18:45:54 UTC
(In reply to comment #15)
> 
> It might, if you collect traces both with and without the autoexposure option
> being on and mail them to the driver author / maintainer Jean-François Moine
> he
> might be able to add such an option under Linux.

Thanks! Sent the traces.

Note You need to log in before you can comment on or make changes to this bug.