[snarfed]capjamesg how you handle exceeding the limit is up to you, but garbage collecting like that is dangerous if you're discarding user data. again I recommend lazy evaluation and sizing batches empirically at runtime to fit in a given amount of memory
