Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GuineaPig assumes there's only one file needed to uploaded into hdfs in addition to itself, which could cause errors. #1

Open
normanyahq opened this issue Sep 22, 2016 · 0 comments

Comments

@normanyahq
Copy link

normanyahq commented Sep 22, 2016

https://github.com/TeamCohen/GuineaPig/blob/master/guineapig.py#L1360

        self._shippedFiles = []
        self._gpigSourceFile = sys.argv[0]
        self.ship(GPig.MY_LOC)
        self.ship(self._gpigSourceFile)

When I was writing homework2 in GuineaPig, I wrote a separate file in addition to nb.py and import it in the nb.py. However, that file was not uploaded to the server, causing the map-reduce task failure, which was really hard to debug.

A fix would be upload *.py from the directory of sys.argv[0]

BTW, the GPig version on autolab is not 1.3.5, which doesn't contain combiner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant