Skip to content

[join-] concat: store src sheet names in a hidden col#2929

Open
midichef wants to merge 1 commit intosaulpw:developfrom
midichef:concat_sourcenames
Open

[join-] concat: store src sheet names in a hidden col#2929
midichef wants to merge 1 commit intosaulpw:developfrom
midichef:concat_sourcenames

Conversation

@midichef
Copy link
Contributor

@midichef midichef commented Jan 2, 2026

Here's a prototype of a feature for convenience. When joining sheets with concat, add a hidden column with the name of the source sheet.

Should we do the same for full, append, and merge?

@saulpw
Copy link
Owner

saulpw commented Jan 5, 2026

Isn't there already a hidden column origin_sheet on ConcatSheet with the source sheet in it?

@frosencrantz
Copy link
Contributor

It looks like append has it, and I use it all the time with append ConcatSheets. It looks like full and merge sheets do not.

@midichef
Copy link
Contributor Author

midichef commented Jan 8, 2026

Okay, so append gets a origin_sheet, but concat, full, merge and diff don't. I think that column should be added to all of them.

Also, I just noticed that joining a sheet with full or diff fails when there are no key columns on any sheet:

if 0 in nkeys or len(nkeys) != 1:
vd.fail(f'all sheets must have the same number of key columns')

I'd like that join to succeed when all the joined sheets have 0 key columns.

Should I implement both of these?

@saulpw
Copy link
Owner

saulpw commented Jan 8, 2026

Thanks, @midichef, an implementation would be appreciated. Ideally it would work like it does for concat, with the sheet object being stored in the underlying row data structure, and a hidden column providing the sheet object itself (which will display as its name).

I'm not sure what origin_sheet would mean for those other join types (like full), which pull data from multiple sheets. Do you think it should be origin_sheets in those cases, and expose an array of all sheets the data came from?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants