Tim,
> So I've got this stored procedure which takes DECIMAL(18,6) parameters and
> returns DECIMAL(18,12) results, and internally copies stuff into and out of
> DOUBLE PRECISION variables which it uses to call a UDF to do the actual
> calculations.
>
> And the results are different (in the 8th decimal place or so) between
> running on 32 bit and 64 bit systems.
Please post the logic for the SP and a reproducible example, this may be worthy of a bug tracker ticket.
I have confirmed that the versions of the underlying library (GDAL) are different on the two systems, and for that and other reasons that the best use of my time at the moment is doing other things, so I will be assuming that it's the change to the GDAL code that resulted in the slightly different output and will not be researching it any further.
-- Tim Ward
Posted by: Tim Ward <tdw@telensa.com>
| Reply via web post | • | Reply to sender | • | Reply to group | • | Start a New Topic | • | Messages in this topic (9) |
Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu. Try FAQ and other links from the left-side menu there.
Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
No comments:
Post a Comment