Skip to content

Added compiling target Android support #982

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 23 commits into
base: devel
Choose a base branch
from
Open

Added compiling target Android support #982

wants to merge 23 commits into from
+1,363 −770

Conversation

shadichy
Copy link

No description provided.

@kdave kdave force-pushed the devel branch 6 times, most recently from ef43ce6 to 3eff852 Compare May 30, 2025 14:14
@kdave
Copy link
Owner

kdave commented May 30, 2025

Thanks, this will need to be reworked so there are no standalone ifdefs just for android. Compatibility wrappers should be hidden in files, detected at build time and the implementation switched by some ifdef. I'll apply the patches without changes as this is what you've probably tested and do the updates myself.

As this will become supported config, we'll need github actions for that too.

@kdave kdave added this to the v6.15 milestone May 30, 2025
@kdave
Copy link
Owner

kdave commented May 30, 2025

The qsort_r can be emulated by providing a thread-local storage and calling qsort with a stub function that will call the comparator with the stored data. There's more qsort_r usage and likely will be in the future so we should not need to do the emulation everywhere.

kdave
kdave reviewed May 30, 2025
View reviewed changes
pthread/pthread.sym Outdated
pthread_cancel; /* used */
local:
*;
};
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed? The file is not used anywhere.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's for debugging pthread

adam900710 and others added 16 commits June 6, 2025 08:24
@adam900710
btrfs-progs: mkfs: add --inode-flags option
4676c5a 
This new option allows end users to specify certain per-inode flags for
specified file/directory inside rootdir.

And mkfs will follow the kernel behavior by inheriting the inode flag
from the parent.

For example:

 rootdir
 |- file1
 |- file2
 |- dir1/
 |  |- file3
 |- subv/     << will be created as a subvolume using --subvol option
    |- dir2/
    |  |- file4
    |- file5

When `mkfs.btrfs --rootdir rootdir --subvol subv --inode-flags
nodatacow:dir1 --inode-flags nodatacow:subv", then the following files
and directory will have *nodatacow* flag set:

- dir1
- file3
- subv
- dir2
- file4
- file5

For now only two flags are supported:

- nodatacow
  Disable data COW, implies *nodatasum* for regular files

- nodatasum
  Disable data checksum only.

This also works with --compress option, and files with nodatasum or
nodatacow flag will skip compression.

Signed-off-by: Qu Wenruo <[email protected]>
Signed-off-by: David Sterba <[email protected]>
@adam900710
btrfs-progs: tests: new test case for mkfs.btrfs --inode-flags
e3823c6 
The simple test will create a layout like the following:

rootdir
|- file1
|- file2
|- subv/		<< Regular subvolume
|  |- file3
|- nocow_subv/		<< NODATACOW subvolume
|  |- file4
|- nocow_dir/		<< NODATACOW directory
|  |- dir2
|  |  |- file5
|  |- file6
|- nocow_file1		<< NODATACOW file

Any files under NODATACOW subvolume/directory should also be NODATACOW.
The explicitly specified single file should also be NODATACOW.

Issue: kdave#984
Signed-off-by: Qu Wenruo <[email protected]>
Signed-off-by: David Sterba <[email protected]>
@morbidrsa @adam900710
btrfs-progs: zoned: create a data block-group for relocation
deb9200 
Create a second data block-group to be used for relocation, in case a
zoned filesystem in created.

This second data block-group will then be picked up by the kernel as the
default data relocation block-group on mount.

This ensures we always have a target to relocate good data to when we
need to do garbage collection.

Signed-off-by: Johannes Thumshirn <[email protected]>
Signed-off-by: David Sterba <[email protected]>
@maharmstone @adam900710
btrfs-progs: print csum values on superblock mismatch
45a237b 
If in btrfs_check_super() we find that the superblock has a csum
mismatch, print the wanted and found values, just as we do for metadata
in __csum_tree_block_size().

When hex-editing a btrfs image, it's useful to use btrfs check to
calculate what the new csum should be. Unfortunately at present this
only works for trees and not for the superblock, meaning you have to use
the much more wordy `btrfs inspect-internal`.

Pull-request: kdave#985
Signed-off-by: Mark Harmstone <[email protected]>
Signed-off-by: David Sterba <[email protected]>
@ozraru @adam900710
btrfs-progs: docs: fix kernel version to set defrag compress level
c534971 
This feature is provided by commit of kernel fc5c0c58258748 ("btrfs:
defrag: extend ioctl to accept compression levels") which is not
included in 6.14 but 6.15.

[skip ci]

Pull-request: kdave#983
Signed-off-by: David Sterba <[email protected]>
@adam900710
btrfs-progs: convert: add feature dependency checks for bgt
24b64c1 
Block group tree requires no-holes and free-space-tree features, add
such check just like mkfs.

Signed-off-by: Qu Wenruo <[email protected]>
Signed-off-by: David Sterba <[email protected]>
@adam900710
btrfs-progs: convert: replace the bytenrs check with a UASSERT()
62fd046 
The bytenr sequence of all roots are controlled by our code, so if
something went wrong with the sequence, it's a bug.

A UASSERT() is more suitable for this case.

Signed-off-by: Qu Wenruo <[email protected]>
Signed-off-by: David Sterba <[email protected]>
@adam900710
btrfs-progs: convert: simplify insert_temp_root_item()
e27f4a5 
The function requires parameters @slot and @itemoff to record where the
next item should land.

But this is overkilled, as after inserting an item, the temporary extent
buffer will have its header nritems and the item pointer updated.  We
can use that header nritems and item pointer to get where the next item
should land.

This removes the external counter to record @slot and @itemoff.

Signed-off-by: Qu Wenruo <[email protected]>
Signed-off-by: David Sterba <[email protected]>
@adam900710
btrfs-progs: convert: simplify insert_temp_dev_item() and insert_temp…
9657aff 
…_chunk_item()

These functions require parameters @slot and @itemoff to record where the
next item should land.

But this is overkilled, as after inserting an item, the temporary extent
buffer will have its header nritems and the item pointer updated.

We can use that header nritems and item pointer to get where the next
item should land.

This removes the external counter to record @slot and @itemoff.

Signed-off-by: Qu Wenruo <[email protected]>
Signed-off-by: David Sterba <[email protected]>
@adam900710
btrfs-progs: convert: simplify insert_temp_dev_extent()
3a8d004 
This function requires parameters @slot and @itemoff to record where the
next item should land.

But this is overkilled, as after inserting an item, the temporary extent
buffer will have its header nritems and the item pointer updated.

We can use that header nritems and item pointer to get where the next
item should land.

This removes the external counter to record @slot and @itemoff.

Signed-off-by: Qu Wenruo <[email protected]>
Signed-off-by: David Sterba <[email protected]>
@adam900710
btrfs-progs: convert: simplify insert_temp_extent_item() and insert_t…
fb030a7 
…emp_block_group()

These functions require parameters @slot and @itemoff to record where the
next item should land.

But this is overkilled, as after inserting an item, the temporary extent
buffer will have its header nritems and the item pointer updated.

We can use that header nritems and item pointer to get where the next
item should land.

This removes the external counter to record @slot and @itemoff.

Signed-off-by: Qu Wenruo <[email protected]>
Signed-off-by: David Sterba <[email protected]>
@adam900710
btrfs-progs: convert: merge setup_temp_fs_tree() and setup_temp_csum_…
a26657d 
…tree()

Both fs and csum trees are empty at make_convert_btrfs(), no need to use
two different functions to do that.

Merge them into a common setup_temp_empty_tree() instead.

Signed-off-by: Qu Wenruo <[email protected]>
Signed-off-by: David Sterba <[email protected]>
@adam900710
btrfs-progs: convert: implement the block group tree support properly
15f0b6a 
Previously there were some problems related to btrfs-convert bgt support,
that it doesn't work at all, caused by the following reasons:

- We never update the super block with extra compat ro flags
  Even if we set "-O bgt" flags, it will not set the compat ro flags,
  and everything just go non-bgt routine.

  Meanwhile other compat ro flags are for free-space-tree, and
  free-space-tree is rebuilt after the full convert is done.
  Thus this bug won't cause any problem for fst features, but only
  affecting bgt so far.

- No extra handling to create block group tree

Fix above problems by:

- Set the proper compat RO flag for the temporary super block
  We should only set the compat RO flags except the two FST related
  bits.  As FST is handled after conversion, we should not set the flag
  at that timing.

- Add block group tree root item and its backrefs
  So the initial temporary fs will have a proper block group tree.

  The only tricky part is for the extent tree population, where we have
  to put all block group items into the block group tree other than the
  extent tree.

With these two points addressed, now block group tree can be properly
enabled for btrfs-convert.

Signed-off-by: Qu Wenruo <[email protected]>
Signed-off-by: David Sterba <[email protected]>
@adam900710
btrfs-progs: tests: tests: add a test case for convert with bgt feature
9a29f0c 
Previously "btrfs-convert -O bgt" would not cause any error, but the
resulting fs has no block-group-tree feature at all, making it no
different than "btrfs-convert -O ^bgt".

This is a big bug that was never caught by our existing convert runs.
001-ext2-basic and 003-ext4-basic all tested bgt feature, but don't
really check if the resulting fs really have bgt flags set.

To fix that add a new test case, which will do the regular bgt convert,
but at the end also do a super block dump and verify the
BLOCK_GROUP_TREE flag is properly set.

Signed-off-by: Qu Wenruo <[email protected]>
Signed-off-by: David Sterba <[email protected]>
@kdave @adam900710
btrfs-progs: docs: add 6.15 kernel development statistics
932205b 
[ci skip]

Signed-off-by: David Sterba <[email protected]>
@kdave @adam900710
btrfs-progs: docs: update 6.15 contribution graphs
96d5e91 
[ci skip]

Signed-off-by: David Sterba <[email protected]>
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 11, 2025
View reviewed changes
shadichy added 2 commits June 11, 2025 20:23
@shadichy
btrfs-progs: Add pthread_cancel implementation for bionic
eda8ec8 
Created `pthread/btrfs_pthread.h` and `pthread/btrfs_pthread.c` to allow Android and bionic libc emulating `pthread_setcanceltype`, `pthread_setcancelstate`, `pthread_cancel` and some other definitions.
Since Android NDK does not support `pthread_cancel()`, this workaround method uses `pthread_kill()` instead.

Signed-off-by: Shadichy <[email protected]>
@shadichy
btrfs-progs: rescue-chunk-recover: Added atomic patch
0ffc07e 
This patch is from termux/termux-packages

Signed-off-by: Shadichy <[email protected]>
shadichy added 2 commits June 11, 2025 20:23
@shadichy
btrfs-progs: list-chunks: Workarounds for replacing qsort_r when ta…
6f9271f 
…rgeting Android

Android NDK does not support `qsort_r`, this workaround first add a list of `id` to struct `compare`, and then reverse the list, and finally do the sorting stuffs using `qsort`.
This helps preserve the sorting order without `qsort_r`

Signed-off-by: Shadichy <[email protected]>
@shadichy
btrfs-progs: pthread: Removed pthread.sym as it is not used anywhere
a0de648 
Signed-off-by: Shadichy <[email protected]>
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 12, 2025
View reviewed changes
android-compat/qsort.c
ctx.compar = compar;
ctx.arg = arg;
struct qsort_r_context *old_ctx = qsort_r_ctx;
qsort_r_ctx = &ctx;

Check warning

Code scanning / CodeQL

Local variable address stored in non-local memory Warning

A stack address (
source
Loading
) may be assigned to a non-local variable.
@kdave kdave force-pushed the devel branch 2 times, most recently from 0e9536e to 8f2c556 Compare June 20, 2025 17:29
@kdave kdave modified the milestones: v6.15, v6.16 Jun 20, 2025
@kdave kdave force-pushed the devel branch 2 times, most recently from dafafca to 5d47f58 Compare June 20, 2025 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kdave kdave kdave left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v6.16
Development

Successfully merging this pull request may close these issues.
6 participants
@shadichy @kdave @adam900710 @morbidrsa @maharmstone @ozraru