Hy will treat a Hy file as Python if it doesn't have a ".hy" file extension

[Kodiologist]

$ echo '(print 1 2)' >testcode
$ hy testcode
Traceback (most recent call last):
  File "/usr/local/bin/hy", line 12, in <module>
    sys.exit(hy_main())
  File "/usr/local/lib/python3.7/site-packages/hy/cmdline.py", line 369, in hy_main
    sys.exit(cmdline_handler("hy", sys.argv))
  File "/usr/local/lib/python3.7/site-packages/hy/cmdline.py", line 355, in cmdline_handler
    runpy.run_path(filename, run_name='__main__')
  File "/usr/local/lib/python3.7/runpy.py", line 261, in run_path
    code, fname = _get_code_from_file(run_name, path_name)
  File "/usr/local/lib/python3.7/site-packages/hy/importer.py", line 195, in _get_code_from_file
    code = compile(source, fname, 'exec')
  File "testcode", line 1
    (print 1 2)
           ^
SyntaxError: invalid syntax

@brandonwillard, you might want to take a look at this one.

[brandonwillard]

Ah, yes, all the Hy-specific loader logic from #1672 is based on file extensions and falls back to Python builtins otherwise. In other words, it only enforces a choice between files recognized as Hy and Python.

Do we want to assume that extension-less files and/or unrecognized extensions are Hy? Should we raise an exception (from the cmdline tool) when we don't find/recognize an extension? Should we fail silently in those cases?

We can make some fairly straightforward changes to produce most of those conditions, but we probably want to be more laissez-faire than not. Python's choice to interpret every file as Python source seems sorta reasonable if only because it handles the entire import process — in which we are simply injecting Hy parsing capabilities. If we attempt to take over default behaviours, we might need to do a lot more just to preserve expected functionality elsewhere (e.g. make sure we don't block other importlib finders and loaders).

[refi64]

I think anything passed via the command line should be automatically assumed to be a Hy module. Other than that, the import rules could probably stay intact.

[Kodiologist]

Yeah, I think it's reasonable to expect a .hy file extension of Hy files that are imported, but hy SOMETHING should always interpret SOMETHING as Hy—unless, perhaps, it both has a file extension and that extension is .py or .pyc.

[Kodiologist]

We also presumably want shebang lines to work:

$ printf '#!/usr/bin/env hy\n(print 1 2)\n' > myprog
$ chmod 755 myprog                                  
$ ./myprog
Traceback (most recent call last):
  …
  File "./myprog", line 2
    (print 1 2)
           ^
SyntaxError: invalid syntax

[brandonwillard]

@kirbyfan64, that's the approach I was initially leaning toward, and it would require us to either stop using runpy in cmdline.py or force the runpy patch to use our loader. Both of those approaches avoid changing the importer/finder we put on sys.path_hooks and negatively affecting the basic import process (e.g. making it greedy or blocking downstream importers/finders); however, they also effectively impose their own greedy import process just for the cmdline and/or runpy.

@Kodiologist, handling a shebang is much more in-line with the approach I'm currently leaning toward, which attempts to determine whether or not a file is Hy source through extension and (simple) non-extension means. Such a change could be introduced in the way described above, or more generally through changes to the importer/finder. Since we implement the Python 2.7 importer/finder, it's clear where/how we should do this, but, with our minimally-invasive Python 3.x patch, we might need to do some research if a simple change to this line in _hy_source_to_code isn't enough.

Also, it's quite possible I'm over-thinking this, and that simply changing the extension conditions in the 2.7 loader, importer/finder and 3.x loader patch will work just fine. Still, I think it's safer to define a clear and constrained — albeit potentially non-standard (e.g. exclusively extension, shebang, and perhaps light syntax inference) — means of determining when a file is Hy source.

[refi64]

Well IMO if someone's using Hy to run a non-Hy file, they're doing something very wrong...

[brandonwillard]

Even so, we don't want to break valid Hy code that triggers imports or runpy invocations of other file types. That's my main concern.

[Kodiologist]

We should probably avoid trying to interpret shebang lines (that's the shell's job) or detect Hy syntax. I think that hy FILENAME will need an exception from the usual import rules, and perhaps it shouldn't use runpy.

[brandonwillard]

All right, I'll start a PR for changes to that effect.