c# - Memory confusion -
i have app in i'm trying create large "cube" of bytes. 3 dimensional array (~1000x1000x500) saves of values i'm interested in - i'm getting out of memory problems. while expected, behavior of various oom messages i'm getting has been quite confusing. first:
foo[,,] foo1 = new foo[1000, 1000, 500];
fails oom error, not:
foo[,,] foo1 = new foo[250, 1000, 500];
foo[,,] foo2 = new foo[250, 1000, 500];
foo[,,] foo3 = new foo[250, 1000, 500];
foo[,,] foo4 = new foo[250, 1000, 500];
shouldn't these 2 sets of code consume same amount of memory?
also, getting error message when ~1.5gb had been consumed, assumed switching 64bit application let me use more memory before failing.
am running stack space limitations? , if so, how can instantiate structure entirely on heap without ever having exist (as single entity) on stack?
thanks in advance - forward light can shead on behavior.
edit
i musing on more featured implementation of answer, thought i'd append. i'm not sure if parallelization help, perhaps depends on initializer
.
using system; using system.linq; public static t[][][] newjagged<t>( int h, int w, ing d, func<int, int, int, t> initializer = null, bool parallelize = true) { if (h < 1) { throw new argumentoutofrangeexception("h", h, "dimension less 1.") } if (w < 1) { throw new argumentoutofrangeexception("w", w, "dimension less 1.") } if (d < 1) { throw new argumentoutofrangeexception("d", d, "dimension less 1.") } if (initializer == null) { initializer = (i, j, k) => default(t); } if (parallelize) { return newjaggedparalellimpl(h, w, d, initializer); } return newjaggedimpl(h, w, d, initializer); } private static t[][][] newjaggedimpl<t>( int h, int w, int d, func<int, int, int, t> initializer) { var result = new t[h][][]; (var = 0; < h; i++) { result[i] = new t[w][]; (var j = 0; j < w; j++) { result[i][j] = new t[d]; (var k = 0; k < d; k++) { result[i][j][k] = initializer(i, j, k); } } } return result; } private static t[][][] newjaggedparalellimpl<t>( int h, int w, int d, func<int, int, int, t> initializer) { var result = new t[h][][]; parallelenumerable.range(0, h).forall(i => { result[i] = new t[w][]; parallelenumerable.range(0, w).forall(j => { result[i][j] = new t[d]; parallelenumerable.range(0, d).forall(k => { result[i][j][k] = initializer(i, j, k); }); }); }); return result; }
this makes function generic still leaves simple syntax,
var foo1 = newjagged<foo>(1000, 1000, 500);
you fancy , populate in paralell on initialization,
var foo2 = newjagged<foo>( 1000, 1000, 5000, (i, j, k) => { var pos = (i * 1000 * 500) + (j * 500) + k; return ((pos % 2) == 0) ? new foo() : null; });
in instance, populating checkerboard effect (i think.);
this may not seem answer problem ...
if had function, this
public static t[][][] threedimmer<t>(int h, int w, int d) t : new() { var result = new t[h][][]; (var = 0; < h; i++) { result[i] = new t[w][]; (var j = 0; j < w; j++) { result[i][j] = new t[d]; (var k = 0; k < d; k++) { result[i][j][k] = new t(); } } } return result; }
then have encapsulated initialization of 3 dimensional jagged array of reference types. allow do,
foo[][][] foo1 = threedimmer<foo>(1000, 1000, 500);
this avoid memory fragmentation issues of multidimensional arrays. avoid other pitfalls , limitations, giving faster more flexible jagged array instead.
Comments
Post a Comment