Monday, 18 January 2016

RE: [firebird-support] 32 bit vs 64 bit arthmetic

 

Tim,

> So I've got this stored procedure which takes DECIMAL(18,6) parameters and
> returns DECIMAL(18,12) results, and internally copies stuff into and out of
> DOUBLE PRECISION variables which it uses to call a UDF to do the actual
> calculations.
>
> And the results are different (in the 8th decimal place or so) between
> running on 32 bit and 64 bit systems.

Please post the logic for the SP and a reproducible example, this may be worthy of a bug tracker ticket.


Sean

__._,_.___

Posted by: "Leyne, Sean" <Sean@broadviewsoftware.com>
Reply via web post Reply to sender Reply to group Start a New Topic Messages in this topic (8)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Visit http://www.firebirdsql.org and click the Documentation item
on the main (top) menu.  Try FAQ and other links from the left-side menu there.

Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

.

__,_._,___

No comments:

Post a Comment