Build only php -S, passthru(), redirect to server from symlink

[guest271314]

Description

I only require built-in local server php -S localhost:8000 and passthru(), not the remainder of PHP.

Additionally I need to make requests to the local server using a symbolic link.

Are the above two requirements possible?

<?php 
  header('Vary: Origin');
  header("Access-Control-Allow-Origin: *");
  header("Access-Control-Allow-Methods: GET");
  header("Content-Type: application/octet-stream");
  header("X-Powered-By:");
  echo passthru("parec -d @DEFAULT_MONITOR@");
  exit();
}

let abortable = new AbortController();
let {signal} = abortable;
// 'chrome-extension://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/test.txt' 
// is symbolic link to http://localhost:8000
fetch('chrome-extension://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/test.txt', {signal})
.then((r) => r.body)
.then((readable) => readable.pipeTo(new WritableStream({
  write(v) {console.log(v) // do stuff},
  close() {console.log('Stream closed.')}
}))
.catch(console.warn);

[damianwadley]

I can't tell what you're doing or why you say that the extension resource is a "symbolic link" to a URL, but I doubt PHP is getting in the way of whatever it is.

What do you expect the browser to do and what is the browser actually doing?

[KapitanOczywisty]

PHP is not good for such tasks, but since you know something about JS, you should use NodeJS.

This is proof of concept, but it's place to start:

const http = require("http");
const { spawn } = require("child_process");

/** @type {http.RequestListener} */
const requestListener = function (req, res) {
  console.log("client connected");

  res.setHeader("Vary", "Origin");
  res.setHeader("Access-Control-Allow-Origin", "*");
  res.setHeader("Access-Control-Allow-Methods", "GET");
  res.setHeader("Content-Type", "application/octet-stream");
  res.writeHead(200);

  const cmd = spawn("parec", ["-d", "@DEFAULT_MONITOR@"]);

  cmd.stdout.on("data", (chunk) => {
    console.log("cmd data " + chunk.length);
    res.write(chunk);
  });

  req.on("close", () => {
    console.log("client end");
    cmd.kill();
  });

  cmd.on("close", () => {
    console.log("cmd end");
    res.end();
  });
};

const server = http.createServer(requestListener);
server.listen(8080);

And run with:

node ./index.js

https://nodejs.org/api/child_process.html#child_processspawncommand-args-options
https://nodejs.org/api/http.html#httpcreateserveroptions-requestlistener

[guest271314]

I can't tell what you're doing

Executing arbitrary shell scripts on any web page.

In this case streaming live system audio output to speakers ("What-U-Hear") to the web page caller. Consider a jam-session that can be streamed to other peers and saved locally, an internet radio station where the source is the audio I am playing on mpv, or the browser itself. Chrome restricts capture of monitor devices on *nix deliberately, so it is not possible to capture audio output to speakers or headphones that are not being played in a Chrome tab, i.e., using navigator.mediaDevices.getDisplayMedia({audio: true, video: true}) or navigator.mediaDevices.getUserMedia({audio: true}), which are microphone only capture devices on *nix.

Browser extensions have a "web_accessible_resources" key where sites matched can request the local resource bypassing CORS and CSP restrictions.

  "web_accessible_resources": [{
      "resources": [ "*.html", "*.js", "*.svg", "*.png", "*.php", "*.txt"],
      "matches": [ "<all_urls>" ],
      "extensions": [ ]
  }],

For example, GitHub serves CSP headers that prevent requesting http://localhost:8000 with fetch(), errors will be thrown.

When fetch('chrome-extension://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/test.txt') is called on GitHub, e.g., at console or in an <iframe> the extension inserts into the page the response is served, though only the static file.

I can start and stop the local server using Native Messaging, for example https://github.com/guest271314/native-messaging-espeak-ng/blob/master/local_server.sh

#!/bin/bash
# https://stackoverflow.com/a/24777120
send_message() {
  message="$1"
  # Calculate the byte size of the string.
  # NOTE: This assumes that byte length is identical to the string length!
  # Do not use multibyte (unicode) characters, escape them instead, e.g.
  # message='"Some unicode character:\u1234"'
  messagelen=${#message}
  # Convert to an integer in native byte order.
  # If you see an error message in Chrome's stdout with
  # "Native Messaging host tried sending a message that is ... bytes long.",
  # then just swap the order, i.e. messagelen1 <-> messagelen4 and
  # messagelen2 <-> messagelen3
  messagelen1=$(( ($messagelen      ) & 0xFF ))               
  messagelen2=$(( ($messagelen >>  8) & 0xFF ))               
  messagelen3=$(( ($messagelen >> 16) & 0xFF ))               
  messagelen4=$(( ($messagelen >> 24) & 0xFF ))               
  # Print the message byte length followed by the actual message.
  printf "$(printf '\\x%x\\x%x\\x%x\\x%x' \
        $messagelen1 $messagelpen2 $messagelen3 $messagelen4)%s" "$message"
}
local_server() {
  if pgrep -f 'php -S localhost:8000' > /dev/null; then
    pkill -f 'php -S localhost:8000' & send_message '"Local server off."' 
  else
    php -S localhost:8000 & send_message '"Local server on."'
  fi
}
local_server

https://github.com/guest271314/native-messaging-espeak-ng/blob/master/nativeTransferableStream.js

onload = async () => {
  chrome.runtime.sendNativeMessage(
    'native_messaging_espeakng',
    {},
    async (nativeMessage) => {
      parent.postMessage(nativeMessage, name);
      await new Promise((resolve) => setTimeout(resolve, 100));
      const controller = new AbortController();
      const { signal } = controller;
      parent.postMessage('Ready.', name);
      onmessage = async (e) => {
        if (e.data instanceof ReadableStream) {
          try {
            const { value: file, done } = await e.data.getReader().read();
            const fd = new FormData();
            const stdin = await file.text();
            fd.append(file.name, stdin);
            const { body } = await fetch('http://localhost:8000', {
              method: 'post',
              cache: 'no-store',
              credentials: 'omit',
              body: fd,
              signal,
            });
            parent.postMessage(body, name, [body]);
          } catch (err) {
            parent.postMessage(err, name);
          }
        } else {
          if (e.data === 'Done writing input stream.') {
            chrome.runtime.sendNativeMessage(
              'native_messaging_espeakng',
              {},
              (nativeMessage) => {
                parent.postMessage(nativeMessage, name);
              }
            );
          }
          if (e.data === 'Abort.') {
            controller.abort();
          }
        }
      };
    }
  );
};

where here the output of espeak-ng is captured and streamed to browser https://github.com/guest271314/native-messaging-espeak-ng/blob/master/index.php, again, to workaround the fact that Chromium does not support capture of window.speechSythesis.speak() audio output; the audio is not played on the Chromium tab, rather at the system level, why this code https://stackoverflow.com/a/70665493 does not work as expected (see above re getDisplayMedia({audio: true, video: true}) Tab capture; https://github.com/guest271314/captureSystemAudio#background).

<?php 
if (isset($_POST["espeakng"])) {
    header('Vary: Origin');
    header("Access-Control-Allow-Origin: chrome-extension://<id>");
    header("Access-Control-Allow-Methods: POST");
    header("Content-Type: application/octet-stream");
    header("X-Powered-By:");
    echo passthru($_POST["espeakng"]);
    exit();
  }

Currently I am streaming using Native Messaging, using the <iframe> approach, in pertinent part https://github.com/guest271314/captureSystemAudio/blob/master/native_messaging/capture_system_audio/capture_system_audio.py

 while True:
        receivedMessage = getMessage()
        process = subprocess.Popen(split(receivedMessage), stdout=subprocess.PIPE)
        os.set_blocking(process.stdout.fileno(), False)
        for chunk in iter(lambda: process.stdout.read(1024 * 1024), b''):
            if chunk is not None:
                encoded = str([int('%02X' % i, 16) for i in chunk])
                sendMessage(encodeMessage(encoded))                       

https://github.com/guest271314/captureSystemAudio/blob/master/native_messaging/capture_system_audio/transferableStream.js

async function handleMessage(value, port) {
    try {
      await writer.ready;
      await writer.write(new Uint8Array(JSON.parse(value)));
    } catch (e) {
      console.error(e.message);
    }
    return true;
}

https://github.com/guest271314/captureSystemAudio/blob/master/native_messaging/capture_system_audio/audioStream.js

async nativeMessageStream() {
    return new Promise((resolve) => {
      onmessage = (e) => {
        if (e.origin === this.src.origin) {
          console.log(e.data);
          if (!this.source) {
            this.source = e.source;
          }
          if (e.data === 1) {
            this.source.postMessage(
              { type: 'start', message: this.stdin },
              '*'
            );
          }
          if (e.data === 0) {
            document
              .querySelectorAll(`[src="${this.src.href}"]`)
              .forEach((f) => {
                document.body.removeChild(f);
              });
            onmessage = null;
          }
          if (e.data instanceof ReadableStream) {
            this.stdout = e.data;
            resolve(this.captureSystemAudio());
          }
        }
      };
      this.transferableWindow = document.createElement('iframe');
      this.transferableWindow.style.display = 'none';
      this.transferableWindow.name = location.href;
      this.transferableWindow.src = this.src.href;
      document.body.appendChild(this.transferableWindow);
    }).catch((err) => {
      throw err;
    });
  }

Ideally I do not want to inject an <iframe> into the arbitrary web page, as when errors occur before the process completes that <iframe> can be still in the page, requiring both page and extension reload to regain expected functionality.

PHP passthru() works for my uses cases, WebTransport is verbose and does not actually support indefinite streaming per my testing https://groups.google.com/a/chromium.org/g/web-transport-dev/c/-nRKS9ws8tc/m/pIAo2gVDAgAJ.

The one caveat that starting and stopping the server I have experienced is the delay between pgrep being called and the server starting, thus the await new Promise((resolve) => setTimeout(resolve, 100)); in nativeTransferableStream.js before fetch() call.

Practically, I don't need the remainder of PHP outside of local server functionality and passthru(), perhaps getmypid() to avoid costly pgrep, if I need to check if server is already running, if I don't just keep the server on. That is why I asked how to build with only php -S and passthru() (exec()) functionality, so I can essentially port that minimal PHP build for my use cases.

PHP is not good for such tasks, but since you know something about JS, you should use NodeJS.

PHP works as expected for my uses cases. See above proofs. From what I gather NodeJS is expensive. I am trying to travel as light as possible. Comparatively what is the cost of that code versus index.php, above? What is the CPU usage comparing https://stackoverflow.com/a/48443161 with https://github.com/simov/native-messaging; and php -S host:port with require("http")? Creating the server is not the issue, accessing the local server on arbitrary web pages is.

I started diving into PAC files https://bugs.chromium.org/p/chromium/issues/detail?id=839566#c40 after I filed this issue, which might be an additional alternative solution.

I want to be able to request the ''chrome-extension://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/test.txt'' and get the response from the PHP development server that I start and stop using Native Messaging at arbitrary web pages. Naively I am talking about ln [OPTION]... [-T] TARGET LINK_NAME run in the Chromium extension folder and requests to 'test.txt' being forward to the local PHP server which issues response.

[guest271314]

For clarity,

it is not possible to capture audio output to speakers or headphones that are not being played in a Chrome tab, i.e., using navigator.mediaDevices.getDisplayMedia({audio: true, video: true}) or navigator.mediaDevices.getUserMedia({audio: true}), which are microphone only capture devices on *nix.

is not technically correct, it is possible, just without user-defined code guest271314/SpeechSynthesisRecorder#17. The last time I checked the quality of audio output by WebRTC was sub-par compared to processing the raw PCM directly then using AudioContext and MediaStreamTrackGenerator for output.

[guest271314]

This is not a bug. This is a feature request. A very specific question about whether it is possible, or not, to build a minimal PHP with only php -S host:port ans passthru() capability.

The redirect to server from symbolic link is a secondary question.

[KapitanOczywisty]

From what I gather NodeJS is expensive.

NodeJS is faster and lighter than php. Especially when dealing with shell and sockets. Also php -S is intended only for testing, not for anything near production use. And for you it might be better to use the same language for server and client part. If you need a bit more complex server: use fastify

[guest271314]

NodeJS is faster and lighter than php. Especially when dealing with shell and sockets.

What is the basis of that claim?

Empirical evidence?

The last time I checked running NodeJS was around 25MB, PHP 18MB. Python Native Messaging host uses 12MB. That is why I use a Python Native Messaging host. You can run the PHP version https://stackoverflow.com/a/48443161 and the NodeJS version https://github.com/simov/native-messaging and Python version mdn/webextensions-examples#157 report the MB used per your OS's Task Manager equivalent here.

That is not really germain to my inquiry.

Also php -S is intended only for testing, not for anything near production use.

My machine, and any user that employs the pattern is the production.

I don't follow rules. If I did I would not be capturing monitor devices or keeping ServiceWorker active indefinitely on Chromium.

"Obey no rules, I'm doing me " -Talk To Me, Run The Jewels

How to achieve the stated goals of this feature request?

[nikic]

This is not a bug. This is a feature request. A very specific question about whether it is possible, or not, to build a minimal PHP with only php -S host:port ans passthru() capability.

Okay, this is pretty easy to answer: No, this is not possible. A minimal build of PHP includes multiple extensions (core, standard, spl, date, json, hash, maybe more I'm forgetting) and I don't believe we have an interest in providing a more minimal build. We want that applications and libraries can depend on certain core functionality being available.

[guest271314]

@nikic And the second question?

[guest271314]

@nikic Did you miss the 2d question in this issue?

The redirect to server from symbolic link is a secondary question.

[KapitanOczywisty]

If you're talking about proxying requests, php -S doesn't have that. With apache you have mod_proxy, nginx - reverse proxy, with build-in php server you have to handle request on your own, e.g. with curl.

As for minimal build, I'm assuming you're already compiling php with ./configure --disable-all, with that you will have only the following modules:

./sapi/cli/php -m
[PHP Modules]
Core
date
hash
json
pcre
Reflection
SPL
standard

[damianwadley]

Doesn't sound like a question of PHP doing any proxying:

I want to be able to request the ''chrome-extension://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/test.txt'' and get the response from the PHP development server that I start and stop using Native Messaging at arbitrary web pages. Naively I am talking about ln [OPTION]... [-T] TARGET LINK_NAME run in the Chromium extension folder and requests to 'test.txt' being forward to the local PHP server which issues response.

You can't symlink a file to a webpage, nor can you symlink a webpage to a webpage. That's just not how symlinks work. So that's not going to work.

If you want to request some resource within a Chrome extension and have the browser redirect to a website then you'll have to find some Chrome extension-based mechanism to do so. PHP has nothing to do with this.

[guest271314]

@damianwadley It's not a web page. It is a local file. A locally unpacked extension is a local folder consisting of a manifest.json file, and/or other files. Those files are mapped to the extension URL chrome-extension://<id>/test.txt. When I am on example.com and click the extension icon the only files involved or executed are local files. The redirection from the local file test.txt would be to the local php -S localhost not redirection to any external website. Everything is local. The sound emitting from the local system, e.g. PulseAudio, the php -S builtin-server, the Chromium extension,the test.txt file. I have done something similar with inotify-tools that emits events for open, close, modify. I was asking if when I do $ ln -s test.txt localserver that localserver be an alias for, or be redirected to the local php -S instance running in the same local directory. Someone else asking a similar question on stackoverflow which was never answered. The system crashed during experiments, I lost the bookmark.

Native Messaging approach uses less local resources than local server. However, connecting from an arbitrary web page to Native Messaging is not direct. I will probably stay with that approach. I was simply preforming due diligence in determining if a concept is possible: fetching local file (or symlink), which I can already do on any site, and redirecting the local file (or symlink) request to the php built-in local server. A modest proposal.

In any event, I will continue trying to manifest what I am trying to achieve myself. Thanks.

[damianwadley]

It's not a web page. It is a local file. A locally unpacked extension is a local folder consisting of a manifest.json file, and/or other files. Those files are mapped to the extension URL chrome-extension://<id>/test.txt.

Yes, they may exist as local files, but what I mean is that they are not treated as local files. The security policy is a little different, but for most intents and purposes, chrome-extension resources look and act the same way as static files hosted on a website.

[guest271314]

@damianwadley FWIW I figured out how to achieve an appreciable part of the requirement.

First I remove Content-Security-Policy header from all sites I navigate to https://github.com/guest271314/remove-csp-header which allows me to reuest localhost from any origin.

Second I use Native Messaging to start PHP built-in server

#!/usr/bin/env php
<?php
function server() {
  passthru("php -S localhost:8000 index.php");
}

function out($data = "") {
  $fp = fopen("php://stdout", "w");
  if ($fp) {
    server();
    $on = array("message" => "Local server running.");
    $on_message = json_encode($on);
    //Send the length of data
    fwrite($fp, pack("L", strlen($on_message)));
    fwrite($fp, $on_message);
    fflush($fp);
    fclose($fp);
    exit(0);
  } else {
    exit(1);
  }
}

function err($data) {
  $fp = fopen("php://stderr", "w");
  if ($fp) {
    fwrite($fp, $data);
    fflush($fp);
    fclose($fp);
  }
  return;
}

function in() {
  $data = "";
  $fp = fopen("php://stdin", "r");
  if ($fp) {
    //Read first 4 bytes as unsigned integer
    $len = current(unpack("L", fread($fp, 4)));
    $data = fread($fp, $len);
    fclose($fp);
  } else {
    exit(1);
  }
  return $data;
}

while (true) {
 if (($l = in()) !== "") {
    out($l);
  } else {
    exit(0);
  }
}
exit(0);
?>

index.php

<?php
    header("Access-Control-Allow-Origin: *");
    header("Access-Control-Allow-Methods: GET, POST");
    header("Access-Control-Allow-Headers: Access-Control-Request-Private-Network");
    header("Access-Control-Allow-Private-Network: true");
    header("Content-Type: application/octet-stream");
    header("Vary: Origin");
    echo passthru($_POST["capture_system_audio"]);
    print(getmypid());
    print(connection_aborted());
    if (connection_aborted() != 0) {
      die();
    }
?>

What I am having challenge with now is automatically terminating the local server when the request is aborted using AbortController.abort().

var fd = new FormData();
var abortable = new AbortController();

fd.append('capture_system_audio', 'parec -d @DEFAULT_MONITOR@');
fetch('http://localhost:8000', {
  method: 'post',
  signal: abortable.signal,
  cache: 'no-store',
  body: fd,
  headers: { 'Access-Control-Request-Private-Network': true },
})
  .then((r) => r.body)
  .then((b) =>
    b.pipeTo(
      new WritableStream({
        abort(reason) {
          console.log(reason);
        },
        close() {
          console.log('closed');
        },
        write(v) {
          console.log(v);
        },
      })
    )
  )
  .catch(console.error);

When I want to about the otherwise never-ending request

abortable.abort();

I have tried

passthru("kill -9 " . getmypid());

and

 if (connection_aborted() != 0) {
      die();
 }

after the

echo passthru("command");

line.

At terminal when the request is aborted I observe

[Sat Feb 12 08:49:16 2022] 127.0.0.1:38670 [200]: POST /
write() failed: Broken pipe
[Sat Feb 12 08:49:16 2022] 127.0.0.1:38670 Closing

printed.

How can I catch the

write() failed: Broken pipe

or

Closing

and terminate/kill the current built-in server process?

[damianwadley]

@guest271314 If you kill PHP after the audio is finished then you'll have to start it up manually the next time. That doesn't make sense to me. If what you want is to stop PHP from running, and therefore prevent the audio thing from working until you start up PHP again, then control that separately so there's no accidents.