Loop over the allocated blocks

[ksaurabh-amd]

I want to loop over the allocated blocks and see how much of the memory is in each of the block is allocated. One of the thing that I want to do is pack the memory together and prevent usage of small memory location in the block.

In the first one you have 4 blocks that have some memory regions that are allocated. Because of this the block cannot be freed. But by arranging the memory we can get only one occupied block and rest 3 can be freed. I wanted to achieve something like this.
(Blue region shows where the memory is allocated)

Is there any suggestion or a way in which I can loop over the blocks.

[MrBurmark]

This is an interesting idea. I have wondered if it would be possible to repack long lived data together to help umpire use less memory or de-fragment. I don't think umpire has considered doing this kind of thing because the calling code has pointers into the allocations and needs to manage the lifetime of the data in those allocations.
Right now you could go over each of your allocations in a pool and allocate a new buffer, copy your data, and free the original buffer. That would hopefully place the new allocations in a more compact arrangement but there is no guarantee of that.
What about an approach where umpire manages the loop over its internal data structures and decides what it would make sense to move? I can imagine a de-fragmentation interface where you could tell umpire which allocations can be moved, it could decide which to move, and then you could provide a callback to copy/move the data and update the pointers in the calling code.

[ksaurabh-amd]

Hi,

Thanks for the response. Basically I would like to group the memory for certain blocks that has very low usable memory together.

frag_mem = 0
for i = 1:nblocks
    f = compute fraction of usable memory in each block
    if(f < some_percent): // say 20% (basically 20% of GPU has usable memory and rest 80% can be released)
        frag_mem += usable_mem(block)
    endif
endfor

new_mem = allocate(frag_mem)
for i = 1:nblocks
    f = compute fraction of usable memory in each block
    if(f < some_percent):
        move data from block to new_mem
        free(memory used by block)
    endif
endfor

I will try copying data in new buffer, as you suggested but something in umpire interface will greatly help.

Thanks,