ProcessBuilder.Source.#> is nonblocking from 2.12.1

[hongxuchen]

It seems that from 2.12.1 ProcessBuilder.Source.#> begins working asynchronously, which conflicts with the statements:

!: blocks until all external commands exit, and returns the exit code of the last one in the chain of execution.
!!: blocks until all external commands exit, and returns a String with the output generated.

Welcome to Scala 2.12.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_112).
Type in expressions for evaluation. Or try :help.

scala> import scala.language.postfixOps
import scala.language.postfixOps

scala> import sys.process._
import sys.process._

scala> import java.net.URL
import java.net.URL

scala> import java.io.File
import java.io.File

// big file, about ~1.5G
scala> val s = "http://mirror.0x.sg/ubuntu-releases/16.04.2/ubuntu-16.04.2-desktop-amd64.iso"
s: String = http://mirror.0x.sg/ubuntu-releases/16.04.2/ubuntu-16.04.2-desktop-amd64.iso

scala> val pb = new URL(s) #> new File("u16042.iso")
pb: scala.sys.process.ProcessBuilder =  ( http://mirror.0x.sg/ubuntu-releases/16.04.2/ubuntu-16.04.2-desktop-amd64.iso #| /home/hongxu/src/puzzles/u16042.iso )

scala> pb !
res0: Int = 0  // <--- return code printed out almost immediately

Real external commands like s"curl ${s}" ! is working correctly. #10055 mentioned a similar issue of ProcessBuilder.Sink.#< but was introduced in 2.12.0 .

[som-snytt]

I can't reproduce the result of the REPL printing a res0 result.

Try -Xprint:typer to see exactly what you're compiling. Maybe there's something in your local context.

[hongxuchen]

Hi @som-snytt , I'm not quite familiar with scala REPL, so I just paste my result when running scala -Xprint:typer for last two statements.

(I just noticed that pb ! wasn't in the snippet I pasted above, sorry for the mistake.)

scala> val pb = new URL(s) #> new File("u16042.iso")
[[syntax trees at end of                     typer]] // <console>
package $line17 {
  object $read extends scala.AnyRef {
    def <init>(): $line17.$read.type = {
      $read.super.<init>();
      ()
    };
    object $iw extends scala.AnyRef {
      def <init>(): type = {
        $iw.super.<init>();
        () pb !
      };
      object $iw extends scala.AnyRef {
        def <init>(): type = {
          $iw.super.<init>();
          ()
        };
        import scala.sys.process._;
        object $iw extends scala.AnyRef {
          def <init>(): type = {
            $iw.super.<init>();
            ()
          };
          import scala.language.postfixOps;
          import $line14.$read.$iw.$iw.$iw.$iw.s;
          import java.io.File;
          import java.net.URL;
          object $iw extends scala.AnyRef {
            def <init>(): type = {
              $iw.super.<init>();
              ()
            };
            private[this] val pb: scala.sys.process.ProcessBuilder = scala.sys.process.`package`.urlToProcess(new java.net.URL($line14.$read.$iw.$iw.$iw.$iw.s)).#>(new java.io.File("u16042.is
o"));
            <stable> <accessor> def pb: scala.sys.process.ProcessBuilder = $iw.this.pb
          }
        }
      }
    }
  }
}

[[syntax trees at end of                     typer]] // <console>
package $line17 {
  object $eval extends scala.AnyRef {
    def <init>(): $line17.$eval.type = {
      $eval.super.<init>();
      ()
    };
    <stable> <accessor> lazy val $result: scala.sys.process.ProcessBuilder = $line17.$read.$iw.$iw.$iw.$iw.pb;
    <stable> <accessor> lazy val $print: String = {
      $line17.$read.$iw.$iw.$iw.$iw;
      "\u001B[1m\u001B[34mpb\u001B[0m: \u001B[1m\u001B[32mscala.sys.process.ProcessBuilder\u001B[0m = ".+(scala.runtime.ScalaRunTime.replStringOf($line17.$read.$iw.$iw.$iw.$iw.pb, 1000))
    }
  }
}

#######################################################################################################

scala> pb !

[[syntax trees at end of                     typer]] // <console>
package $line18 {
  object $read extends scala.AnyRef {
    def <init>(): $line18.$read.type = {
      $read.super.<init>();
      ()
    };
    object $iw extends scala.AnyRef {
      def <init>(): type = {
        $iw.super.<init>();
        ()
      };
      object $iw extends scala.AnyRef {
        def <init>(): type = {
          $iw.super.<init>();
          ()
        };
        import scala.sys.process._;
        object $iw extends scala.AnyRef {
          def <init>(): type = {
            $iw.super.<init>();
            ()
          };
          import scala.language.postfixOps;
          import java.io.File;
          import java.net.URL;
          import $line17.$read.$iw.$iw.$iw.$iw.pb;
          object $iw extends scala.AnyRef {
            def <init>(): type = {
              $iw.super.<init>();
              ()
            };
            private[this] val res0: Int = $line17.$read.$iw.$iw.$iw.$iw.pb.!;
            <stable> <accessor> def res0: Int = $iw.this.res0
          }
        }
      }
    }
  }
}

[[syntax trees at end of                     typer]] // <console>
package $line18 {
  object $eval extends scala.AnyRef {
    def <init>(): $line18.$eval.type = {
      $eval.super.<init>();
      ()
    };
    <stable> <accessor> lazy val $result: Int = $line18.$read.$iw.$iw.$iw.$iw.res0;
    <stable> <accessor> lazy val $print: String = {
      $line18.$read.$iw.$iw.$iw.$iw;
      "\u001B[1m\u001B[34mres0\u001B[0m: \u001B[1m\u001B[32mInt\u001B[0m = ".+(scala.runtime.ScalaRunTime.replStringOf($line18.$read.$iw.$iw.$iw.$iw.res0, 1000))
    }
  }
}

[chengpohi]

Hi @som-snytt and @hongxuchen , I have tried to look at this issue, this issue can be reproduce in Scala 2.12.1 and Scala 2.12.2, and create a PR scala/scala#5918 to try to fix this issue. Could you help review?

[hongxuchen]

@chengpohi 👍 Thanks so much !

The fix looks good to me. But as I'm not able to build the source and try now, so I cannot help much.

Anyway, I'll close this issue and let professional reviewers judge your PR 😄

[som-snytt]

Thanks, I've helped look at the process code before, so I'll take a look. Let's leave the issue open until a fix is merged.