Home » Developer & Programmer » JDeveloper, Java & XML » BigDecimal issue
BigDecimal issue [message #545682] Thu, 01 March 2012 11:57 Go to next message
aucrun
Messages: 114
Registered: February 2011
Senior Member
Hi,

I'm having an problem using BigDecimal Library for Javascript.

Imagine I have the following value:
3.438470

This evil number is giving me the headaches, because I 'm not able to round it as I need.
Using the BigDecimal library I am not able to round it to 3 decimal places correctly. It allways rounds it to 3.438 instead of the 3.439 value it should return.

There are two major rounding modes I'm using:
- ROUND_UP
- ROUND_HALF_UP

The first solves this particular problem but gives me another one, since, as described in its documentation (here)
it will never decrements the original value, and it the original number changes to 3.438440 it would return 3.439 also.

The second gives me allways 3.438 because, as written in its documentation it only sees if the marginal numbers to the right of the Decimal Places are bigger or smaller than 0.5
this is wrong, because, in this particular case, 3.438470 should be:
3.43847
3.4385
and finally 3.439

Anyone has an idea of how to solve this?

thakns!
Re: BigDecimal issue [message #546747 is a reply to message #545682] Thu, 08 March 2012 09:16 Go to previous message
aucrun
Messages: 114
Registered: February 2011
Senior Member
Hi again,

I've actually found out a way of doing it.

We can iterate the SetPlaces n times, accordingly with the decimal places to the right of the required decimal place.
In the example:
Value to Round - 3.438447
Rounded Value from C# (decimal datatype) - 3.438
Rounded Value from BigDecimal (Round_Half_Up) - 3.438
Regular Round Value from Excel - 3.438
Required Result - 3.439

Using the BigDecimal setScale we can do this:
var bd_number= new BigDecimal("3.438447");
var result = bd_number.setScale(4, BigDecimal.prototype.ROUND_HALF_UP).setScale(3, BigDecimal.prototype.ROUND_HALF_UP);

I think, that the main problem is that, with huge numbers the amount of iterations may consume a lot of processor resources!

It works for me, since I only need to look for the first 6 decimal places to the right of zero! and round only by the first 3!
Previous Topic: How to get the metadata (in xml format) of all the fileds in SQl query ?
Next Topic: 'Invalid Column Index' error - Java Web applications
Goto Forum:
  


Current Time: Thu Mar 28 08:50:04 CDT 2024