Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improper parsing]: Hackerone #129

Open
abhinavsecond opened this issue Oct 12, 2023 · 6 comments
Open

[Improper parsing]: Hackerone #129

abhinavsecond opened this issue Oct 12, 2023 · 6 comments
Assignees
Labels
Bug Something isn't working Effort: 1 week The implementation should require about one week Performance Things are unreasonably slow

Comments

@abhinavsecond
Copy link

InQL version

BApp Store

GraphQL API

https://hackerone.com/graphql

GraphQL specification version

No response

What isn't working?

I have waited more that 30+ min to show the results of HackerOne Graphql schemas but it is not showing any result
Note:- You need to be authenticated to query graphql on hackerone
image
It is stuck a this point

@abhinavsecond
Copy link
Author

Well after 10 more mins the result finally appeared but blank
image

@execveat
Copy link
Contributor

Thank you for the report! The Activity query alone takes 2 minutes to get parsed on my machine. I filed the issue with GQLSpection, will take a look at optimizations we could add here: doyensec/GQLSpection#32

The next InQL version contains syntax highlighting which causes some additional drop in performance, so we definitely need to address it.

@execveat execveat added Bug Something isn't working Effort: 1 week The implementation should require about one week labels Oct 12, 2023
@abhinavsecond
Copy link
Author

Thanks for looking into it🙂
I also noticed that while using this, it's using a lot of resources in Burp Suite, up to 7.2 GB of RAM.
image
I was running it on my main computer, which has plenty of RAM, so it didn't crash Burp. If it was on a virtual machine (VM), I think it might have caused the VM to crash.
I believe you should investigate this further because even though it's a great tool, if it crashes a user's Burp Suite or VM, they won't be able to use it effectively.

Thanks
Abhinav

@execveat execveat added the Performance Things are unreasonably slow label Oct 20, 2023
@execveat
Copy link
Contributor

So, this turned out to be way more complicated than I initially thought. I'm planning to address the performance issue by reducing recursion to an iterative loop and generating queries on demand (so, only when you click one, not ahead of time). Initially this was planned for a future release, but it looks like a release breaking issue now.

In case of H1 specifically, some queries right now are more than 1GB in size which explains the observed RAM usage. Obviously, this size makes them completely impractical to work with. So we could just add a sanity check that detects these huge queries and refuses to work with them, but obviously these problematic queries might very well be the most interesting ones, so in my view you should be able to work with them using Burp & InQL (on a beefy machine).

So, it's a complicated problem that will likely need to be addressed both from performance tuning perspective (everything's single core right now...) as well as from the UI/UX to intelligently disable heavyweight features such as syntax highlighting and inline comments for larger queries. Stay tuned.

@execveat
Copy link
Contributor

I replaced the recursion with a loop and the parsing time for the whole h1 schema (on my machine) now is under 2 minutes. I will try to reduce this loading time further by generating queries on demand (upon the click) instead of ahead-of-time, as well as introducing parallelization.

You can try the new version, but it's not release ready yet. Clicking larger queries (like activity) completely freezes Burp, presumably due to expensive highlighting routine. I'll try to introduce auto-disabling highlighting for larger queries as a stop-gap measure, followed by performance optimization.

@marz-hunter
Copy link

I also experienced this problem. I load from introspection (json file) takes -+ 45 minutes. but when clicking to view it can't be seen (I'm using the version from the baap store 10 oct 2023)

bartek-doyensec added a commit that referenced this issue Mar 17, 2025
* First Kotlin-rewritten version

* Removed references to Burp's legacy API

* Upgrade gradlew wrapper

* Bump upgraded features for compatibility with the future Gradle & Java

* Add pre-commit linters

* Fix style issues suggested by ktlint

* Add embedded GraphiQL #111

* Add GraphQL Playground and GraphQL Voyager

* Add "Send to" buttons in GraphiQL

* Refactoring build scripts to use Taskfile for parallelization

* Eject graphiql

* Eject Playground

* Eject Voyager

* Move web apps rebuilds from Gradle to Taskfile

* Enable output from the build process

* Fix a bug that reset File to null after first failure.

* Fix the argument parsing to support arguments with the whitespace

* Lower the watch interval by default

* Modify right click handlers to support configuration through InQL Settings

* Expose embedded web IDEs in the context menu

* Fixes #132

* Fixes #127

* Pull in updated GQLSpection for faster parsing (#129)

* A bunch of minor performance improvements #129

* Try introspection query from latest to earliest - Closes #134

* Optimized Taskfile

* Added node_modules to .gitignore

* Updated build.gradle

* Refactored embedded webserver, fixed CORS and Introspection cache

* Taskfile: add default task

* Implemented accurate embedded browser opening

* Implemented External Tool Webserver Lazy Loading

* Made some Chromium args dynamic based on current env

* Added missing item in context menu items list

* Improved GraphQL request detection

* Added "Save to file" action

* Fixed some warnings

* Made sendToEmbeddedToolActions a list

* Fixed getGraphQLQuery and reorganized classes in packages

* Implemented Proxy Request Highlighter

* Fixed some more minor warnings

* Injecting Profile's customheaders in requests from inql.burp

* Minor optimizations

* Fixed settings and added new features options

* Formatting is now async and cached to improve performance

* Targeting latest GQLSpection commit

* Updated GraphiQL yarn checksums

* Fixed yarn concurrency issues

* Improved params sent to external IDEs

* Added option to strip comments when sending query from Scanner Results

* Chromium: don't include args if dir not found

* Exclude Origin from Session to allow CORS fixing

* Polishing for the release

* Linked to latest GQLSpection commit

* Handle empty queries and mutations

* Removing Altair and Playground

* Updated GQLSpection to latest commit

* Fixing GraphiQL sending to repeater and intruter

* Polishing the UI

* Fixing active tab bottom border

* Adding support for darkmode

* Setting new tab title

* Upgraded dependencies

* Updated contributors

* Remove python3 linting stuff

* Fixing tab coloring bug

* Editable tabs style fixes

* Use GQLSpektion

* Rename "Attacker" tab to "Batch Queries"

* Integrated GraphQL native parsing

* Restore data from project file

* Fixed glitchy JTree UI

* Batch queries view improvements

* Add changeListener

* Fixing Batch UI

* Fixing Batch UI

* Adding POI

* Adding POI

* Adding POI

* Adding POI

* Adding Cycles Detection

* Tiny improvements

* Removing "not implemented yet" text from settings

* Comment

* Changing Scanner view sections %

* Fixing formatted editor scroll speed

* Adding information about object truncating

* Fixing exception when formating queries

* Hardcoding jar name

* adding .kotlin

* Description and others before release to 6.0

---------

Co-authored-by: lokiuox <[email protected]>
Co-authored-by: Andrew Konstantinov <[email protected]>
Co-authored-by: Savio Sisco <[email protected]>
bartek-doyensec added a commit that referenced this issue Mar 17, 2025
* First Kotlin-rewritten version

* Removed references to Burp's legacy API

* Upgrade gradlew wrapper

* Bump upgraded features for compatibility with the future Gradle & Java

* Add pre-commit linters

* Fix style issues suggested by ktlint

* Add embedded GraphiQL #111

* Add GraphQL Playground and GraphQL Voyager

* Add "Send to" buttons in GraphiQL

* Refactoring build scripts to use Taskfile for parallelization

* Eject graphiql

* Eject Playground

* Eject Voyager

* Move web apps rebuilds from Gradle to Taskfile

* Enable output from the build process

* Fix a bug that reset File to null after first failure.

* Fix the argument parsing to support arguments with the whitespace

* Lower the watch interval by default

* Modify right click handlers to support configuration through InQL Settings

* Expose embedded web IDEs in the context menu

* Fixes #132

* Fixes #127

* Pull in updated GQLSpection for faster parsing (#129)

* A bunch of minor performance improvements #129

* Try introspection query from latest to earliest - Closes #134

* Optimized Taskfile

* Added node_modules to .gitignore

* Updated build.gradle

* Refactored embedded webserver, fixed CORS and Introspection cache

* Taskfile: add default task

* Implemented accurate embedded browser opening

* Implemented External Tool Webserver Lazy Loading

* Made some Chromium args dynamic based on current env

* Added missing item in context menu items list

* Improved GraphQL request detection

* Added "Save to file" action

* Fixed some warnings

* Made sendToEmbeddedToolActions a list

* Fixed getGraphQLQuery and reorganized classes in packages

* Implemented Proxy Request Highlighter

* Fixed some more minor warnings

* Injecting Profile's customheaders in requests from inql.burp

* Minor optimizations

* Fixed settings and added new features options

* Formatting is now async and cached to improve performance

* Targeting latest GQLSpection commit

* Updated GraphiQL yarn checksums

* Fixed yarn concurrency issues

* Improved params sent to external IDEs

* Added option to strip comments when sending query from Scanner Results

* Chromium: don't include args if dir not found

* Exclude Origin from Session to allow CORS fixing

* Polishing for the release

* Linked to latest GQLSpection commit

* Handle empty queries and mutations

* Removing Altair and Playground

* Updated GQLSpection to latest commit

* Fixing GraphiQL sending to repeater and intruter

* Polishing the UI

* Fixing active tab bottom border

* Adding support for darkmode

* Setting new tab title

* Upgraded dependencies

* Updated contributors

* Remove python3 linting stuff

* Fixing tab coloring bug

* Editable tabs style fixes

* Use GQLSpektion

* Rename "Attacker" tab to "Batch Queries"

* Integrated GraphQL native parsing

* Restore data from project file

* Fixed glitchy JTree UI

* Batch queries view improvements

* Add changeListener

* Fixing Batch UI

* Fixing Batch UI

* Adding POI

* Adding POI

* Adding POI

* Adding POI

* Adding Cycles Detection

* Tiny improvements

* Removing "not implemented yet" text from settings

* Comment

* Changing Scanner view sections %

* Fixing formatted editor scroll speed

* Adding information about object truncating

* Fixing exception when formating queries

* Hardcoding jar name

* adding .kotlin

* Description and others before release to 6.0

* Replacing HTML with textPane attempt

* New GraphQL Editor WIP

* Bump Montoya API lib to access new features

* Fix wrong text color

* Fix font

* Fix context menu

* Fix close button

* Fix dark mode icons

* Fix directory icon not shown

* Fix context menu

* Fix style cache

* Fix scroll speed

---------

Co-authored-by: lokiuox <[email protected]>
Co-authored-by: Andrew Konstantinov <[email protected]>
Co-authored-by: Savio Sisco <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working Effort: 1 week The implementation should require about one week Performance Things are unreasonably slow
Projects
None yet
Development

No branches or pull requests

3 participants