leaking sv when perl_clone is used after some code is run

[p5pRT]

Migrated from rt.perl.org#29018 (status was 'open')

Searchable as RT29018$

[p5pRT]

From stas@stason.org

Created by stas@rabbit.stason.org

The followinig problem is reproduced with any 5.8-5.9 w/ithreads.

doing the sequence​:

1. create perl
2. eval_pv "sub foo"
3. perl_clone

causes a scalar leak

Here is a program to reproduce​:

/* clone.c */
#include <EXTERN.h>
#include <perl.h>

/*
  * gcc -o clone clone.c `perl-5.8.0-ithread -MExtUtils​::Embed -e ccopts -e
ldopts` -Wall -g
  */

#define TEST "sub foo {}"

int main(int argc, char **argv, char **env)
{
  char *embedding[] = { "", "-le", "0" };
  PerlInterpreter *perl1 = perl_alloc();
  PerlInterpreter *perl2;

  PERL_SET_CONTEXT(perl1);
  perl_construct(perl1);

  perl_parse(perl1, NULL, 3, embedding, (char **)NULL);
  /* DynaLoader must be preloaded before perl_clone, if DynaLoader
  * is to be required later */

  Perl_eval_pv(perl1, TEST, TRUE); /* loaded only by the first perl */

  perl2 = perl_clone(perl1, CLONEf_KEEP_PTR_TABLE);

  PERL_SET_CONTEXT(perl2);
  perl_destruct(perl2);
  perl_free(perl2);

  PERL_SET_CONTEXT(perl1);
  perl_destruct(perl1);
  perl_free(perl1);

  exit(0);
}

The critical part is

#define TEST "sub foo {}"

which gets eval'ed, the problem doesn't appear if some other code is in
there. I've modied the debug code to call sv_dump to the leaked scalar, so
this is what I get when running this program​:

./clone Attempt to free temp
prematurely​: SV 0x8075bd8, Perl interpreter​: 0x8066a10 during global
destruction.
Scalars leaked​: 1
leaked​: 0x8075bd8, Perl interpreter​: 0x8066a10
SV = PVCV(0x8076178) at 0x8075bd8
  REFCNT = 0
  FLAGS = (TEMP,UNIQUE)
  IV = 0
  NV = 0
  COMP_STASH = 0x0
  ROOT = 0x0
  XSUB = 0x0
  XSUBANY = 0
  GVGV​::GV = 0x0
  FILE = "(null)"
  DEPTH = 0
  FLAGS = 0x10
  OUTSIDE_SEQ = 0
  PADLIST = 0x8075be4
  OUTSIDE = 0x8075c08 (UNIQUE)

My guess is that there is a bug in perl_clone(), and something is not properly
cloned.

Unfortunately I wasn't able to come up with a pure perl example to reproduce
the problem. I suppose this is only triggered with embedded perl.

If you are interested where the problem was detected, it's obviously under
mod_perl 2. The following config section triggers this problem under
threaded mpm Apache2​:

<Perl>
sub foo { }
</Perl>

Perl Info

Flags:
     category=core
     severity=high

Site configuration information for perl v5.8.4:

Configured by stas at Fri Apr 16 18:44:17 PDT 2004.

Summary of my perl5 (revision 5 version 8 subversion 4) configuration:
   Platform:
     osname=linux, osvers=2.6.3-7mdk, archname=i686-linux-thread-multi
     uname='linux rabbit.stason.org 2.6.3-7mdk #1 wed mar 17 15:56:42 cet 2004 
i686 unknown unknown gnulinux '
     config_args='-des -Dprefix=/home/stas/perl/5.8.4-ithread -Dusethreads 
-Doptimize=-g -Duseshrplib -Dusedevel -Accflags=-DDEBUG_LEAKING_SCALARS'
     hint=recommended, useposix=true, d_sigaction=define
     usethreads=define use5005threads=undef useithreads=define 
usemultiplicity=define
     useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
     use64bitint=undef use64bitall=undef uselongdouble=undef
     usemymalloc=n, bincompat5005=undef
   Compiler:
     cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS 
-DDEBUG_LEAKING_SCALARS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include 
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
     optimize='-g',
     cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS 
-DDEBUG_LEAKING_SCALARS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include 
-I/usr/include/gdbm'
     ccversion='', gccversion='3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)', 
gccosandvers=''
     intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
     d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
     ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', 
lseeksize=8
     alignbytes=4, prototype=define
   Linker and Libraries:
     ld='cc', ldflags =' -L/usr/local/lib'
     libpth=/usr/local/lib /lib /usr/lib
     libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
     perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
     libc=/lib/libc-2.3.3.so, so=so, useshrplib=true, libperl=libperl.so
     gnulibc_version='2.3.3'
   Dynamic Linking:
     dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E 
-Wl,-rpath,/home/stas/perl/5.8.4-ithread/lib/5.8.4/i686-linux-thread-multi/CORE'
     cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
     RC2


@INC for perl v5.8.4:
     /home/stas/perl/5.8.4-ithread/lib/5.8.4/i686-linux-thread-multi
     /home/stas/perl/5.8.4-ithread/lib/5.8.4
     /home/stas/perl/5.8.4-ithread/lib/site_perl/5.8.4/i686-linux-thread-multi
     /home/stas/perl/5.8.4-ithread/lib/site_perl/5.8.4
     /home/stas/perl/5.8.4-ithread/lib/site_perl
     .


Environment for perl v5.8.4:
     HOME=/home/stas
     LANG=en_GB
     LANGUAGE=en_GB:en
     LC_ADDRESS=en_CA
     LC_COLLATE=en_GB
     LC_CTYPE=en_GB
     LC_IDENTIFICATION=en_CA
     LC_MEASUREMENT=en_CA
     LC_MESSAGES=en_GB
     LC_MONETARY=en_CA
     LC_NAME=en_CA
     LC_NUMERIC=en_CA
     LC_PAPER=en_CA
     LC_TELEPHONE=en_CA
     LC_TIME=en_GB
     LD_LIBRARY_PATH (unset)
     LOGDIR (unset)
 
PATH=/usr//bin:/bin:/usr/bin:.:/usr/local/bin:/usr/X11R6/bin/:/usr/games:/home/stas/bin:/home/stas/bin:/usr/local/bin:/usr/X11R6/bin:/usr/java/j2re1.4.0/bin/
     PERLDOC_PAGER=less -R
     PERL_BADLANG (unset)
     SHELL=/bin/tcsh


-- 
__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com

[p5pRT]

From @iabyn

On Wed, Apr 21, 2004 at 03​:17​:39AM -0000, Stas Bekman wrote​:

doing the sequence​:

1. create perl
2. eval_pv "sub foo"
3. perl_clone

causes a scalar leak

(Technically not a leak!)

The cause of this is as follows​:
At the end of the eval, the CV used to hold the body of the evalled code
is mortalised (so that it's normally freed at the end of the statement
holding the eval). This means that the TEMP flag is set on it, and a
pointer to it is pushed on the tmps stack.

Of course, there is no 'end of statement' in this code, so in the
single-threaded case, the tmps stack would be popped instead in
perl_destruct().

In your code, when you clone the interpreter, the tmps stack isn't cloned
(because the CLONEf_COPY_STACKS flag isn't passed to perl_clone()).
So when the second thread is destructed, the CV isn't freed by the tmps
stack being popped, instead it is freed by the general cleanup - at which
point Perl notices that it still has the TEMP flag set and complains.

One way to fix this is to add a ENTER/SAVETMPS and FREETMPS/LEAVE
to the body of Perl_eval_pv, but I'm not sure if this is the best
solution.

Dave.

Here is a program to reproduce​:

/* clone.c */
#include <EXTERN.h>
#include <perl.h>

/*
* gcc -o clone clone.c `perl-5.8.0-ithread -MExtUtils​::Embed -e ccopts -e
ldopts` -Wall -g
*/

#define TEST "sub foo {}"

int main(int argc, char **argv, char **env)
{
char *embedding[] = { "", "-le", "0" };
PerlInterpreter *perl1 = perl_alloc();
PerlInterpreter *perl2;

  PERL\_SET\_CONTEXT\(perl1\);
  perl\_construct\(perl1\);

  perl\_parse\(perl1\, NULL\, 3\, embedding\, \(char \*\*\)NULL\);
  /\* DynaLoader must be preloaded before perl\_clone\, if DynaLoader
   \* is to be required later \*/

  Perl\_eval\_pv\(perl1\, TEST\, TRUE\); /\* loaded only by the first perl \*/

  perl2 = perl\_clone\(perl1\, CLONEf\_KEEP\_PTR\_TABLE\);

  PERL\_SET\_CONTEXT\(perl2\);
  perl\_destruct\(perl2\);
  perl\_free\(perl2\);

  PERL\_SET\_CONTEXT\(perl1\);
  perl\_destruct\(perl1\);
  perl\_free\(perl1\);

  exit\(0\);

}

The critical part is

#define TEST "sub foo {}"

which gets eval'ed, the problem doesn't appear if some other code is in
there. I've modied the debug code to call sv_dump to the leaked scalar, so
this is what I get when running this program​:

./clone Attempt to free temp
prematurely​: SV 0x8075bd8, Perl interpreter​: 0x8066a10 during global
destruction.
Scalars leaked​: 1
leaked​: 0x8075bd8, Perl interpreter​: 0x8066a10
SV = PVCV(0x8076178) at 0x8075bd8
REFCNT = 0
FLAGS = (TEMP,UNIQUE)
IV = 0
NV = 0
COMP_STASH = 0x0
ROOT = 0x0
XSUB = 0x0
XSUBANY = 0
GVGV​::GV = 0x0
FILE = "(null)"
DEPTH = 0
FLAGS = 0x10
OUTSIDE_SEQ = 0
PADLIST = 0x8075be4
OUTSIDE = 0x8075c08 (UNIQUE)

My guess is that there is a bug in perl_clone(), and something is not properly
cloned.

Unfortunately I wasn't able to come up with a pure perl example to reproduce
the problem. I suppose this is only triggered with embedded perl.

If you are interested where the problem was detected, it's obviously under
mod_perl 2. The following config section triggers this problem under
threaded mpm Apache2​:

<Perl>
sub foo { }
</Perl>

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=high
---
Site configuration information for perl v5.8.4​:

Configured by stas at Fri Apr 16 18​:44​:17 PDT 2004.

Summary of my perl5 (revision 5 version 8 subversion 4) configuration​:
Platform​:
osname=linux, osvers=2.6.3-7mdk, archname=i686-linux-thread-multi
uname='linux rabbit.stason.org 2.6.3-7mdk #1 wed mar 17 15​:56​:42 cet 2004
i686 unknown unknown gnulinux '
config_args='-des -Dprefix=/home/stas/perl/5.8.4-ithread -Dusethreads
-Doptimize=-g -Duseshrplib -Dusedevel -Accflags=-DDEBUG_LEAKING_SCALARS'
hint=recommended, useposix=true, d_sigaction=define
usethreads=define use5005threads=undef useithreads=define
usemultiplicity=define
useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
-DDEBUG_LEAKING_SCALARS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
optimize='-g',
cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
-DDEBUG_LEAKING_SCALARS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include
-I/usr/include/gdbm'
ccversion='', gccversion='3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)',
gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries​:
ld='cc', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=/lib/libc-2.3.3.so, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.3.3'
Dynamic Linking​:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E
-Wl,-rpath,/home/stas/perl/5.8.4-ithread/lib/5.8.4/i686-linux-thread-multi/CORE'
cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches​:
RC2

---
@​INC for perl v5.8.4​:
/home/stas/perl/5.8.4-ithread/lib/5.8.4/i686-linux-thread-multi
/home/stas/perl/5.8.4-ithread/lib/5.8.4
/home/stas/perl/5.8.4-ithread/lib/site_perl/5.8.4/i686-linux-thread-multi
/home/stas/perl/5.8.4-ithread/lib/site_perl/5.8.4
/home/stas/perl/5.8.4-ithread/lib/site_perl
.

---
Environment for perl v5.8.4​:
HOME=/home/stas
LANG=en_GB
LANGUAGE=en_GB​:en
LC_ADDRESS=en_CA
LC_COLLATE=en_GB
LC_CTYPE=en_GB
LC_IDENTIFICATION=en_CA
LC_MEASUREMENT=en_CA
LC_MESSAGES=en_GB
LC_MONETARY=en_CA
LC_NAME=en_CA
LC_NUMERIC=en_CA
LC_PAPER=en_CA
LC_TELEPHONE=en_CA
LC_TIME=en_GB
LD_LIBRARY_PATH (unset)
LOGDIR (unset)

PATH=/usr//bin​:/bin​:/usr/bin​:.​:/usr/local/bin​:/usr/X11R6/bin/​:/usr/games​:/home/stas/bin​:/home/stas/bin​:/usr/local/bin​:/usr/X11R6/bin​:/usr/java/j2re1.4.0/bin/
PERLDOC_PAGER=less -R
PERL_BADLANG (unset)
SHELL=/bin/tcsh

--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http​://stason.org/ mod_perl Guide ---> http​://perl.apache.org
mailto​:stas@​stason.org http​://use.perl.org http​://apacheweek.com
http​://modperlbook.org http​://apache.org http​://ticketmaster.com

--
My get-up-and-go just got up and went.

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From stas@stason.org

Dave Mitchell via RT wrote​:

On Wed, Apr 21, 2004 at 03​:17​:39AM -0000, Stas Bekman wrote​:

doing the sequence​:

1. create perl
2. eval_pv "sub foo"
3. perl_clone

causes a scalar leak

(Technically not a leak!)

:)

Thanks for the research, Dave!

The cause of this is as follows​:
At the end of the eval, the CV used to hold the body of the evalled code
is mortalised (so that it's normally freed at the end of the statement
holding the eval). This means that the TEMP flag is set on it, and a
pointer to it is pushed on the tmps stack.

Of course, there is no 'end of statement' in this code, so in the
single-threaded case, the tmps stack would be popped instead in
perl_destruct().

In your code, when you clone the interpreter, the tmps stack isn't cloned
(because the CLONEf_COPY_STACKS flag isn't passed to perl_clone()).
So when the second thread is destructed, the CV isn't freed by the tmps
stack being popped, instead it is freed by the general cleanup - at which
point Perl notices that it still has the TEMP flag set and complains.

passing CLONEf_COPY_STACKS indeed fixes the problem.

So may be we should pass CLONEf_COPY_STACKS then? Or is it going to
significantly slow down the already so slow cloning process? It's not quite
clear from sv.c description on when that flag should be used​:

  perl_clone takes these flags as parameters​:

  CLONEf_COPY_STACKS - is used to, well, copy the stacks also,
  without it we only clone the data and zero the stacks,
  with it we copy the stacks and the new perl interpreter is
  ready to run at the exact same point as the previous one.
  The pseudo-fork code uses COPY_STACKS while the
  threads->new doesn't.

All I can deduct from this is that if threads->new() don't use it, mod_perl
shouldn't be using it too.

One way to fix this is to add a ENTER/SAVETMPS and FREETMPS/LEAVE
to the body of Perl_eval_pv, but I'm not sure if this is the best
solution.

Well, we can do that locally. The following program works (added the scope
macros and s/perl1/my_perl/ to keep the macros happy)​:

#define TEST "sub foo {}"

int main(int argc, char **argv, char **env)
{
  char *embedding[] = { "", "-le", "0" };
  PerlInterpreter *my_perl = perl_alloc();
  PerlInterpreter *perl2;

  PERL_SET_CONTEXT(my_perl);
  perl_construct(my_perl);

  perl_parse(my_perl, NULL, 3, embedding, (char **)NULL);
  /* DynaLoader must be preloaded before perl_clone, if DynaLoader
  * is to be required later */
  ENTER;SAVETMPS;
  Perl_eval_pv(my_perl, TEST, TRUE); /* loaded only by the first perl */
  FREETMPS;LEAVE;

  perl2 = perl_clone(my_perl, CLONEf_KEEP_PTR_TABLE);

  PERL_SET_CONTEXT(perl2);
  perl_destruct(perl2);
  perl_free(perl2);

  PERL_SET_CONTEXT(my_perl);
  perl_destruct(my_perl);
  perl_free(my_perl);

  exit(0);
}

If that's the case, there should be a note somewhere, that any perl code
running before perl_clone is called must be enclosed in the scope macros? Or
should this be a responsibility of perl to take care of?

--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http​://stason.org/ mod_perl Guide ---> http​://perl.apache.org
mailto​:stas@​stason.org http​://use.perl.org http​://apacheweek.com
http​://modperlbook.org http​://apache.org http​://ticketmaster.com

[p5pRT]

From @iabyn

still present in 5.13.4