The magic size was already bumped from 262 bytes to 8192 bytes at some point, and this is still not enough for e.g. ISO9660, which requires 32k+ bytes to be detected since the first 32k are filled with 00s/vendor defined data irrelevant to the format. Also I've ran into some files that can only be reliably detected by checking the file tail, which easily bumps the "magic size" into megabytes range for no other real benefit.
One way to be able to handle cases like this is having an optional Callable that the user can specify, that will be called by a wrapper if a detector requests more data to make a decision. If the Callable is not specified, the request will automatically be rejected and the detector can fail gracefully.
I'm thinking of something like this:
with open('some_file.bin', 'rb') as f:
def more_bytes(offset: int, size: int) -> bytes:
f.seek(offset)
# The returned number of bytes can be smaller or equal to the requested amount,
# including being 0 bytes long, which can be seen by filetype as a rejection.
return f.read(size)
# The normal magic reading routine here.
f.seek(0)
# If user passes a BufferedReader here, optionally just read from that.
filetype.guess(f.read(8192), more_bytes)
The magic size was already bumped from 262 bytes to 8192 bytes at some point, and this is still not enough for e.g. ISO9660, which requires 32k+ bytes to be detected since the first 32k are filled with 00s/vendor defined data irrelevant to the format. Also I've ran into some files that can only be reliably detected by checking the file tail, which easily bumps the "magic size" into megabytes range for no other real benefit.
One way to be able to handle cases like this is having an optional Callable that the user can specify, that will be called by a wrapper if a detector requests more data to make a decision. If the Callable is not specified, the request will automatically be rejected and the detector can fail gracefully.
I'm thinking of something like this: