IO::File ignores PERL_UNICODE and PerlIO layer defaults [rt.cpan.org #68366]

[toddr]

Migrated from rt.cpan.org#68366 (status was 'new')

Requestors:

• COSMICNET@cpan.org

From cosmicnet@cpan.org on 2011-05-22 02:59:48:

Hi,
  Setting PerlIO defaults with either -C, PERL_UNICODE or the open
pragma, as shown in ${^OPEN}, are complete ignored by IO::File. This
seriously breaks anyones UTF-8 work that uses IO::File, leading to much
frustration.


Lyle

[Leont]

It ignoring the open pragma is expected, as pragmas are lexical.

It ignoring -C and PERL_UNICODE is unexpected and probably a bug.

[toddr]

@cosmicnet see above.

[Leont]

It ignoring -C and PERL_UNICODE is unexpected and probably a bug.

It would appear that both of these only affect the topmost scope, it sets the correct encodings for that lexical scope.

So that would mean this isn't a bug in the IO dist, but in core.

[cosmicnet]

This was 9 years ago, so hard for me to remember exactly. But I do recall messing around with a lot of different stuff with IO layers which just didn't work, when I dug deeper IO::File was ignoring a lot/all of those settings. When I changed to regular open() everything worked as expected.

Unfortunately I don't have scope to put together a test case for this at the moment.

[tonycoz]

It would appear that both of these only affect the topmost scope, it sets the correct encodings for that lexical scope.

So that would mean this isn't a bug in the IO dist, but in core.

It does say:

The C options mean that any subsequent open() (or similar I/O
operations) in the current file scope

but that's pretty vague for a command-line option.

Changing its behaviour to be global would probably break existing code.

[tonycoz]

One way it would be made to work (ignoring the -C issue) is to check the calling scope's $^H{"open<"} and $^H{"open<"} via (caller)[10].

These members of %^H don't seem to be documented, but autodie already uses them.

[Leont]

but that's pretty vague for a command-line option.

Yeah, especially because that's not at all what I would expect it to do.

Changing its behaviour to be global would probably break existing code.

True. TBH the whole concept of -CD is broken anyway if you ask me.

[cosmicnet]

I just came across this in some code I was working on at the time:

my $outfile = new IO::File;
$outfile->open(">$filepath") || die( "Cannot open file '" . decode( 'utf8', $filepath ) . "' for output" );
#$PerlIO::encoding::fallback = Encode::FB_DEFAULT; # This didn't work for some reason
binmode( $outfile, ':encoding(utf8)' );