Visual Studio 2010 C# Double -


i've had problems while converting decimal double in c#. instead of 0.3 getting 0.2999999999999

when debug little bit, i've seen conversion not problem.

see print screen , watch list? http://i.imgur.com/rtudfxo.png

anyone has idea?

edit: answer: strange behavior of visual studio 2010. after restarting fine.

0.3 has infinite representation, when stored binary: (0.) 00111111 11010011 00110011 00110011 00110011 00110011 00110011 00110011 ..... no matter if store 32,64 or 128 bit, cut off @ point.

when converting double loose accuracy. that's why getting 0.2999999999...

this not bug, simple impossible store infinite accuracy.


Comments

Post a Comment

Popular posts from this blog

c# - Operator '==' incompatible with operand types 'Guid' and 'Guid' using DynamicExpression.ParseLambda<T, bool> -

i'm using dynamic linq library , there source code , basic docu , nuget version pm> install-package dynamiclinq i'm trying construct clause involves guids i have tried string "id == @0" . parameter array object[] value ( guid xxxx ) var whereclausesb = buildlogicalkeywhereclause2(entity, logicalkey); //build string var parms = buildparamarray(entity, logicalkey); // object[] var wherelambda = ofsi.bos.core.dynamicexpression.parselambda<t, bool>(whereclausesb.tostring(),parms); //parse an exception thrown in dynamicexpression.parselambda operator '==' incompatible operand types 'guid' , 'guid' i have tried guid , string.(fail) i tried , "id = @0" (fail). string == string works, int32==int32 not guid == guid not any ideas? try using equals method instead of == operator in string: "id.equals(@0)"
Read more