java - Why my JDBC call is consuming memory 4 times more that actual size of data -
i wrote small java program loads data db2 database using simple jdbc call. using select query data , using java statement purpose. have closed statement , connection objects. using 64 bit jvm compilation , running program.
the query returning 52 million records, each row having 24 columns, takes me around 4 minutes load complete data in unix (having multiprocessor environment). using hashmap data-structure load data: map<string, map<string, gridtradestatus>>
. bean gridtradestatus simple getter/setter bean 24 properties in it.
the memory required program alarmingly high. java heap size goes 5.8 - 6gb load complete data while actual used heap size remains between 4.7 - 4.9gb. know should not load data memory business requirements in way only.
the question when put whole data of table in flat file comes out equivalent ~1.2gb. want know why java program consuming memory 4 times more actual size.
there nothing surprising here (to me @ least).
a.) strings in java consume double space compared common text formats (because strings represented utf-16 in heap). also, string object has quite overhead (string object itself, reference char[] contains, hashcode etc.). small strings string object costs memory data contains.
b.) put stuff hashmap. hashmap not memory efficient. first uses default load factor of 75%, means map many entries has big bucket array. then, each entry in map object itself, costs @ least 2 references (key , value) plus object overhead.
in conclusion pretty have expect memory requirements increase quite bit. factor of 4 reasonable if average data string relatively short.
Comments
Post a Comment