PDI allocates new memory that is never freed everytime a variable is exposed

Report by Emilio J. Padrón on Slack:

@Romain and I are using PDI in a long numeric simulation and we have noticed an important memory leak. After a minimal mem profiling I think the issue comes from PDI allocating new memory every time a variable is exposed, memory that is not freed ever. Is it normal (I mean, the expected behavior)?

Just with a small proof of concept I can reproduce this memory leak: (our issue is with a Fortran code, but I obtain the same behavior in C)

#include <pdi.h>

int main(int argc, char* argv[]) {
  PDI_init(PC_parse_path("example6.yml"));

  int my_int = 0;
  float my_float = 0;
  int my_loop = 100000;

  while (my_loop > 0) {
    PDI_multi_expose("event_between",
                     "an_int", &my_int, PDI_OUT,
                     "a_float", &my_float, PDI_OUT,
                     NULL);
    my_int += 1;
    my_float += 0.1;
    my_loop -= 1;
  }

  PDI_finalize();

  return 0;
}

with this simple yaml file:

data:
  an_int: int
  a_float: float

plugins:
  trace: ~

This is a snapshot taken with Massif (Valgrind): screenshot-1634454987

And this is the report from Valgrind Memcheck, warning about the memory being allocated every PDI expose that is never freed:

==366206==
==366206== HEAP SUMMARY:
==366206==     in use at exit: 48,006,358 bytes in 300,048 blocks
==366206==   total heap usage: 7,300,528 allocs, 7,000,480 frees, 829,731,027 bytes allocated
==366206==
==366206== 160 bytes in 1 blocks are possibly lost in loss record 23 of 31
==366206==    at 0x4839F2F: operator new(unsigned long) (vg_replace_malloc.c:417)
==366206==    by 0x493C891: PDI::Scalar_datatype::clone_type() const (in /opt/pdi/lib/libpdi.so.1.4.0-alpha.2021-08-04.09fd084)
==366206==    by 0x48EF431: PDI::Data_descriptor_impl::share(void*, bool, bool) (in /opt/pdi/lib/libpdi.so.1.4.0-alpha.2021-08-04.09fd084)
==366206==    by 0x492C397: PDI_share (in /opt/pdi/lib/libpdi.so.1.4.0-alpha.2021-08-04.09fd084)
==366206==    by 0x492C95B: PDI_multi_expose (in /opt/pdi/lib/libpdi.so.1.4.0-alpha.2021-08-04.09fd084)
==366206==    by 0x109209: main (example6.c:12)
==366206==
==366206== 4,346 (104 direct, 4,242 indirect) bytes in 1 blocks are definitely lost in loss record 29 of 31
==366206==    at 0x48397B5: malloc (vg_replace_malloc.c:380)
==366206==    by 0x496D5A5: PC_parse_file (in /opt/pdi/lib/libparaconf.so.0.4.6)
==366206==    by 0x496D75A: PC_parse_path (in /opt/pdi/lib/libparaconf.so.0.4.6)
==366206==    by 0x109186: main (example6.c:4)
==366206==
==366206== 15,999,840 bytes in 99,999 blocks are definitely lost in loss record 30 of 31
==366206==    at 0x4839F2F: operator new(unsigned long) (vg_replace_malloc.c:417)
==366206==    by 0x493C891: PDI::Scalar_datatype::clone_type() const (in /opt/pdi/lib/libpdi.so.1.4.0-alpha.2021-08-04.09fd084)
==366206==    by 0x48EF431: PDI::Data_descriptor_impl::share(void*, bool, bool) (in /opt/pdi/lib/libpdi.so.1.4.0-alpha.2021-08-04.09fd084)
==366206==    by 0x492C397: PDI_share (in /opt/pdi/lib/libpdi.so.1.4.0-alpha.2021-08-04.09fd084)
==366206==    by 0x492C95B: PDI_multi_expose (in /opt/pdi/lib/libpdi.so.1.4.0-alpha.2021-08-04.09fd084)
==366206==    by 0x109209: main (example6.c:12)
==366206==
==366206== 32,000,000 bytes in 200,000 blocks are definitely lost in loss record 31 of 31
==366206==    at 0x4839F2F: operator new(unsigned long) (vg_replace_malloc.c:417)
==366206==    by 0x493C891: PDI::Scalar_datatype::clone_type() const (in /opt/pdi/lib/libpdi.so.1.4.0-alpha.2021-08-04.09fd084)
==366206==    by 0x48EF431: PDI::Data_descriptor_impl::share(void*, bool, bool) (in /opt/pdi/lib/libpdi.so.1.4.0-alpha.2021-08-04.09fd084)
==366206==    by 0x492C397: PDI_share (in /opt/pdi/lib/libpdi.so.1.4.0-alpha.2021-08-04.09fd084)
==366206==    by 0x492CB82: PDI_multi_expose (in /opt/pdi/lib/libpdi.so.1.4.0-alpha.2021-08-04.09fd084)
==366206==    by 0x109209: main (example6.c:12)
==366206==
==366206== LEAK SUMMARY:
==366206==    definitely lost: 47,999,944 bytes in 300,000 blocks
==366206==    indirectly lost: 4,242 bytes in 39 blocks
==366206==      possibly lost: 160 bytes in 1 blocks
==366206==    still reachable: 1,996 bytes in 7 blocks
==366206==         suppressed: 16 bytes in 1 blocks
==366206== Reachable blocks (those to which a pointer was found) are not shown.
==366206== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==366206==
==366206== For lists of detected and suppressed errors, rerun with: -s
==366206== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 0 from 0)

I am using PDI from the master branch in the repo, specifically commit 09fd0842 from August 4th

Edited by Julien Bigot