Bug 9751 - conversion of denormal floats to double is wrong
Summary: conversion of denormal floats to double is wrong
Status: CLOSED CODE_FIX
Alias: None
Product: Platform Specific/Hardware
Classification: Unclassified
Component: Alpha (show other bugs)
Hardware: All Linux
: P1 normal
Assignee: Richard Henderson
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-01-15 00:43 UTC by Paolo Bonzini
Modified: 2008-06-02 20:56 UTC (History)
1 user (show)

See Also:
Kernel Version: 2.6.18, 2.6.24
Subsystem:
Regression: ---
Bisected commit-id:


Attachments
fix conversion from denormal float to double (749 bytes, patch)
2008-01-15 16:50 UTC, Ivan Kokshaysky
Details | Diff

Description Paolo Bonzini 2008-01-15 00:43:32 UTC
Latest working kernel version: none
Earliest failing kernel version: tested on 2.6.18, but should not work anywhere
Distribution: Debian
Hardware Environment: alphaev56, alphaev68
Software Environment: mathemu=y
Problem Description: conversion from denormal float to double is wrong

Steps to reproduce:

Compile with -O0 -mieee or -O2 -mieee:

#include <stdio.h>
float x = 1.0;
double y;

int g(float z)
{
  printf ("%g\n", z);
}

int main()
{
  x *= (1.0f / 65536.0f);
  x *= (1.0f / 65536.0f);
  x *= (1.0f / 65536.0f);
  x *= (1.0f / 65536.0f);
  x *= (1.0f / 65536.0f);
  x *= (1.0f / 65536.0f);
  x *= (1.0f / 65536.0f);
  x *= (1.0f / 65536.0f);
  x *= (1.0f / 65536.0f);
  x *= (1.0f / 32.0f);
  g(x);
  y = x;
  g(y);
}

Correct value should be:

1.4013e-45
1.4013e-45

Actual value is:

2.65249e-315
2.65249e-315

or

2.65249e-315
0

The right value is 2-149, that is 1.4013e-45, but the alpha does not do it.  The architecture handbook says "A trap handler can convert an S_ floating denormal value into the corresponding T_ floating finite value by adding 896 to the exponent and normalizing", but here 896 is not added (math-emu should, at least on older alphas where it is relevant) and I get a denormal double, 2.65249e-315.

Actually both compiled programs are correct.  You get a zero if the denormal is casted back to float (which makes a zero, because it is way off the range of a float); you get 2.65249e-315 if the compiler performs some copy propagation and sees through the conversion (because if it was not for the errata, the two values should indeed have been equal).

The bug reproduces both on alphaev56 and on alphaev68 (where math-emu should not be relevant, or so I'm told), both with CONFIG_MATHEMU=y.
Comment 1 Paolo Bonzini 2008-01-15 01:16:25 UTC
Where I wrote 2-149, it's 2^-149.  Damn mail client...
Comment 2 Paolo Bonzini 2008-01-15 01:16:39 UTC
Where I wrote 2-149, it's 2^-149.  Damn mail client...
Comment 3 Ivan Kokshaysky 2008-01-15 16:50:51 UTC
Created attachment 14469 [details]
fix conversion from denormal float to double

Good catch, thanks Paolo.

Here is a fix.
Comment 4 Paolo Bonzini 2008-01-15 23:10:57 UTC
Are you sure you don't have to check that the number *is* a denormal?  Especially for older hardware where more math instructions trap to the emulation.
Comment 5 Ivan Kokshaysky 2008-01-16 00:11:52 UTC
Yes, I am. cvtst/s would never trap with finite values. And for NaNs and Infs the patched conversion works correctly.
Comment 6 Paolo Bonzini 2008-01-16 00:14:44 UTC
Thanks!
Comment 7 Natalie Protasevich 2008-06-02 20:56:27 UTC
The fix is in main line, I suppose the bug can be closed.
Thanks.

Note You need to log in before you can comment on or make changes to this bug.