NO_EXIT_RUNTIME by default #5878

kripken · 2017-11-30T03:21:54Z

This makes us not exit the runtime by default. That means we don't emit code for atexits and other things that happen when the runtime shuts down, like flushing the stdio streams. This is beneficial for 2 reasons:

For Better JS size for small programs #5794, this helps remove code. It avoids all the support for shutting down the runtime, emitting atexits, etc. It also enables more optimizations (the ctor evaller in wasm can do better without calls to atexit). This removes 3% of hello world's wasm size and 0.5% of its JS.
A saner default for the web. A program on the web that does anything asynchronous will not want the runtime to exit when main() exits, so we set this flag to 1 for many tests, which this PR lets us remove.

However, this is a breaking change. As already mentioned, the possible breakages are

printf("hello") will not console.log since there is no newline. Only when the streams are flushed would that be printed out. So this change would make us not emit that.
atexits do not run.

Both of those risks are mitigated in this PR:

In ASSERTIONS mode, check if there is unflushed stream output, and explain what to do if so.
Also in ASSERTIONS mode, warn if atexit is called.

This PR has a lot of test changes, some that simplify web tests - because the new default is better for the web - but others that add a param to a shell test - because the new default is less optimal in a shell environment. I think the risk here is lower than those shell tests indicate: we do test quite a lot of things in the shell, but just because it's convenient, not because that's what most users care about.

This PR uncovered 2 existing minor bugs:

An existing bug with our pthreads support: pthreads should always call the shutdown code, even with NO_EXIT_RUNTIME. Otherwise the main thread hangs waiting for them to complete. I think we never noticed this before since we never tested them with NO_EXIT_RUNTIME.
FORCE_FILESYSTEM didn't actually do what the name suggests. Again, I think we just never tested it properly with NO_EXIT_RUNTIME. Fixed in this PR.

…ad in the general case. show warnings in ASSERTIONS mode when exiting the runtime was necessary

… web, actually

…directly, it just happened to work because other things included it

…don't emit it)

…ful for configure/cmake, and easy to do with no overhead).

… to get the exit code from the running program, as if it were a shell command, not a browser app

dschuff · 2017-11-30T22:17:32Z

src/library.js

@@ -596,6 +596,11 @@ LibraryManager.library = {
  atexit__proxy: 'sync',
  atexit__sig: 'ii',
  atexit: function(func, arg) {
+#if ASSERTIONS
+#if NO_EXIT_RUNTIME == 1
+    Runtime.warnOnce('atexit() called, but NO_EXIT_RUNTIME, so atexits() will not be called. set NO_EXIT_RUNTIME to 0');


This could probably use some more detail or a link to a doc. Also in C++, if the user has any static objects with destructors, then those destructors get registered with atexit under the hood. so the doc or message should probably mention that case explicitly.

Good point, added a FAQ entry about that, and mentioned the FAQ in those messages.

dschuff · 2017-11-30T22:22:02Z

src/postamble.js

 #endif
+  if (flush) {


I don't quite understand why we are doing this dance with module.print. If it's ok to just call the users's fflush, why can't we just call it and let it flush for real?

We don't normally even include fflush anymore, we only include it in ASSERTIONS builds. So we don't want to show the proper output here, we want to see if there would be a problem, had we not been in an ASSERTIONS build. So we just check if flushing would have emitting something (and we do so very carefully, as shutting down the FS may throw an error if it hit an error previously or something like that).

I can add more of an explanation to the code?

Reading that again it does seem less than obvious, good point. Added a comment.

…as we leave main(), then do not call exit, we are not exiting yet - code is yet to run later

kripken · 2017-12-01T04:12:38Z

This PR uncovered another existing bug, we used to call exit() when unwinding the stack in emterpreter-async mode. We shouldn't do that as more code will run, main() is not really finished. Fixed in that last commit.

dschuff · 2017-12-05T00:30:09Z

src/library.js

@@ -596,6 +596,11 @@ LibraryManager.library = {
  atexit__proxy: 'sync',
  atexit__sig: 'ii',
  atexit: function(func, arg) {
+#if ASSERTIONS
+#if NO_EXIT_RUNTIME == 1
+    Runtime.warnOnce('atexit() called, but NO_EXIT_RUNTIME, so atexits() will not be called. set NO_EXIT_RUNTIME to 0 (see the FAQ)');


This should probably say 'but NO_EXIT_RUNTIME is 1' (or 'is set') instead of just 'but NO_EXIT_RUNTIME'

dschuff · 2017-12-05T00:30:48Z

src/postamble.js

+#if NO_EXIT_RUNTIME
+      Module.printErr('exit(' + status + ') called, but compiled with NO_EXIT_RUNTIME, so halting execution but not exiting the runtime or preventing further async execution (build with NO_EXIT_RUNTIME=0, if you want a true shutdown)');
+#else
+      Module.printErr('exit(' + status + ') called, but noExitRuntime, so halting execution but not exiting the runtime or preventing further async execution (you can use emscripten_force_exit, if you want to force a true shutdown)');


Thanks, done.

dschuff · 2017-12-05T00:33:54Z

So, I am mostly convinced this is a good idea. I agree that most users would want this behavior and benefit from the size reduction. We currently (with or without this PR) have the situation that if you have NO_EXIT_RUNTIME there's no way to manually cause your destructors to run, right? (i.e. you want to shut down your module and flush, etc)? Has anyone complained about that?

kripken · 2017-12-05T01:14:00Z

We currently (with or without this PR) have the situation that if you have NO_EXIT_RUNTIME there's no way to manually cause your destructors to run, right?

Yes, the mode does not support global destructor calling at all.

Has anyone complained about that?

I can't remember any complaints. But to avoid confusion, we should warn about this in emscripten_force_exit, which advertises itself as being able to do that. I'll fix it.

kripken · 2017-12-05T01:26:45Z

Added that.

kripken · 2017-12-15T23:15:41Z

All feedback above should be addressed, and I have an offline lgtm from @juj, so I'll merge this as soon as tests pass. The last patch updates the metadce test, as after this change is is more effective, and the test expectations need to be updated.

…untime to exit

kripken · 2017-12-16T15:10:05Z

Oddly the pthreads part of browser.test_emscripten_main_loop started to fail. @juj, this is the test we discussed, that we weren't sure why it was passing - now it makes sense and it fails ;)

I made it exit the runtime there, and now it passes. Is that ok?

I also added a commit with a faq entry on this stuff, and a better warning message (that also mentions that emitting a newline is enough to flush std streams).

kripken · 2017-12-16T19:03:47Z

Got on offline ok from @juj, and tests look good.

kripken added 30 commits November 28, 2017 13:15

NO_EXIT_RUNTIME=1 by default: avoid including runtime shutdown overhe…

17ef4e3

…ad in the general case. show warnings in ASSERTIONS mode when exiting the runtime was necessary

fixes

f3eb364

more tests

1106aed

fixes

05f07be

bonus: the new default of NO_EXIT_RUNTIME=1 is the common case on the…

3d623eb

… web, actually

fix

7ca10a1

fix

ac6a066

docs

fa0f631

fix

9e299c5

fix

890c5ea

fix FORCE_FILESYSTEM: it actually never had any code to make it work …

bc3abe8

…directly, it just happened to work because other things included it

fix

1655df2

fixes

268a781

fix

1da0890

fix

b80bdf7

fix

0f6e8fa

better test

2672778

fixes

49a5234

fixes

b8daf47

warn on exitin with a non-zero code when NO_EXIT_RUNTIME (as then we …

ce7b4f4

…don't emit it)

fixes

15a7083

if we exit with a non-zero code, emit it even if NO_EXIT_RUNTIME (use…

1af3ea8

…ful for configure/cmake, and easy to do with no overhead).

fixes

04ce11d

fix flush assertion for c++ code, add testing

6ac52fb

fixes

748f2eb

Merge remote-tracking branch 'origin/incoming' into no-exit-runtime

d2c4f19

update Changelog

0611661

fix

096f7ec

fix browser.test_asm_swapping

e040af1

fix browser.test_emterpreter_async_sleep2_safeheap

39ef944

kripken added 2 commits November 29, 2017 17:23

emrun sets NO_EXIT_RUNTIME=0. it is a mode where we specifically want…

5e81d11

… to get the exit code from the running program, as if it were a shell command, not a browser app

Merge remote-tracking branch 'origin/incoming' into no-exit-runtime

43401c9

dschuff reviewed Nov 30, 2017

View reviewed changes

kripken added 3 commits November 30, 2017 16:44

add faq entry, and mention the faq

72e7255

explain flushing check

4dc9381

fix an existing emterpreter-async bug: if we are unwinding the stack …

6bf8d53

…as we leave main(), then do not call exit, we are not exiting yet - code is yet to run later

dschuff reviewed Dec 5, 2017

View reviewed changes

kripken added 2 commits December 4, 2017 17:06

improve text

195a99c

Merge remote-tracking branch 'origin/incoming' into no-exit-runtime

04c3a9a

kripken added 2 commits December 4, 2017 17:24

warn on calling emscripten_force_exit when NO_EXIT_RUNTIME

be26e6b

whitespace

f6167e7

kripken added 3 commits December 6, 2017 14:28

Merge branch 'incoming' into no-exit-runtime

f8dadc6

Merge branch 'incoming' into no-exit-runtime

9821ad7

metadce is now more effective, update test

9aa1ddb

kripken added 4 commits December 16, 2017 09:04

faq entry on Module.* is not a function [ci skip]

66345ff

Merge branch 'followups' into no-exit-runtime

7d5e86c

improve message

83059d2

fix browser.test_emscripten_main_loop - the pthreads part needs the r…

4299787

…untime to exit

kripken merged commit f585dcb into incoming Dec 16, 2017

kripken deleted the no-exit-runtime branch December 16, 2017 19:06

buu700 mentioned this pull request Dec 27, 2017

Updates for latest emscripten jedisct1/libsodium#653

Merged

curiousdannii mentioned this pull request Jan 14, 2018

Tutorial contains error #6079

Closed

kripken mentioned this pull request Jul 23, 2018

misc testing/CI tidy-up #6890

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NO_EXIT_RUNTIME by default #5878

NO_EXIT_RUNTIME by default #5878

kripken commented Nov 30, 2017

dschuff Nov 30, 2017

kripken Dec 1, 2017

dschuff Nov 30, 2017

kripken Dec 1, 2017 •

edited

Loading

kripken Dec 1, 2017

kripken commented Dec 1, 2017

dschuff Dec 5, 2017

dschuff Dec 5, 2017

kripken Dec 5, 2017

dschuff commented Dec 5, 2017

kripken commented Dec 5, 2017

kripken commented Dec 5, 2017

kripken commented Dec 15, 2017

kripken commented Dec 16, 2017

kripken commented Dec 16, 2017

NO_EXIT_RUNTIME by default #5878

NO_EXIT_RUNTIME by default #5878

Conversation

kripken commented Nov 30, 2017

dschuff Nov 30, 2017

Choose a reason for hiding this comment

kripken Dec 1, 2017

Choose a reason for hiding this comment

dschuff Nov 30, 2017

Choose a reason for hiding this comment

kripken Dec 1, 2017 • edited Loading

Choose a reason for hiding this comment

kripken Dec 1, 2017

Choose a reason for hiding this comment

kripken commented Dec 1, 2017

dschuff Dec 5, 2017

Choose a reason for hiding this comment

dschuff Dec 5, 2017

Choose a reason for hiding this comment

kripken Dec 5, 2017

Choose a reason for hiding this comment

dschuff commented Dec 5, 2017

kripken commented Dec 5, 2017

kripken commented Dec 5, 2017

kripken commented Dec 15, 2017

kripken commented Dec 16, 2017

kripken commented Dec 16, 2017

kripken Dec 1, 2017 •

edited

Loading