Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Paths fail to match on Windows #8

Open
alanisaac opened this issue Jan 4, 2024 · 3 comments · May be fixed by #18
Open

Paths fail to match on Windows #8

alanisaac opened this issue Jan 4, 2024 · 3 comments · May be fixed by #18

Comments

@alanisaac
Copy link
Collaborator

I tried running ocsf-validator on a Windows system recently. I think there's an issue with the way paths are formed where they are not agnostic to the OS.

The way this manifests is that files that should have inferred schema types do not, and the validator eventually fails:

TESTING: Schema types can be inferred
   WARNING: Unable to detect type of events\application\api.json
   WARNING: Unable to detect type of events\application\application.json
   WARNING: Unable to detect type of events\application\application_lifecycle.json
   WARNING: Unable to detect type of events\application\datastore_activity.json
   WARNING: Unable to detect type of events\application\web_resources_activity.json
   WARNING: Unable to detect type of events\application\web_resource_access_activity.json
   WARNING: Unable to detect type of events\base_event.json
   ...

Encountered an unexpected exception:
Traceback (most recent call last):
  File "C:\git\hub\ocsf-validator\ocsf_validator\runner.py", line 245, in validate
    test(
  File "C:\git\hub\ocsf-validator\ocsf_validator\runner.py", line 192, in test
    code()
  File "C:\git\hub\ocsf-validator\ocsf_validator\runner.py", line 247, in <lambda>
    lambda: validate_include_targets(
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\git\hub\ocsf-validator\ocsf_validator\validators.py", line 137, in validate_include_targets
    process_includes(reader, collector=collector, types=types, update=False)
  File "C:\git\hub\ocsf-validator\ocsf_validator\processor.py", line 449, in process_includes
    for target in parser.extract_targets(path):
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\git\hub\ocsf-validator\ocsf_validator\processor.py", line 235, in extract_targets
    if self._types[base] not in [OcsfEvent, OcsfObject]:
       ~~~~~~~~~~~^^^^^^
  File "C:\git\hub\ocsf-validator\ocsf_validator\type_mapping.py", line 29, in __getitem__
    return self._mappings[path]
           ~~~~~~~~~~~~~~^^^^^^
KeyError: 'events\\application\\application.json'

It looks like on Windows, paths are formed with backslashes like objects\user.json whereas the regex matchers are expecting forward slashes e.g. ".*objects/.*json".
@rmouritzen-splunk
Copy link
Contributor

Maybe pathlib globs would be more cross-platform than regex?

@query-jeremy
Copy link
Collaborator

Maybe pathlib globs would be more cross-platform than regex?

... maybe. Some of the matchers use pathlib.Path.match(). It would probably be easier and safer to transform all __getitem__ and __setitem__ keys in Reader and _walk to be platform agnostic and relative to the repo root path. I'll look it over.

@alanisaac alanisaac linked a pull request Apr 2, 2024 that will close this issue
Open
@alanisaac
Copy link
Collaborator Author

I started trying to fix this on top of the tests I added in #18, but found it was a lot of work to refactor all the strings to paths. I might be able to pick this back up later, but anyone is welcome to take a stab!

@alanisaac alanisaac mentioned this issue Apr 19, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[WIP] run unit tests on windows alanisaac/ocsf-validator
3 participants
@alanisaac @query-jeremy @rmouritzen-splunk