Add Signal Messenger Android parser (#1602)#2913
Open
cleitonaugusto wants to merge 5 commits into
Open
Conversation
Extracts individual and group conversations from Signal Android databases
(signal.db) obtained through full-filesystem forensic acquisition.
## Architecture
- SignalParser – Tika AbstractParser; MIME type application/x-signal-db
- SignalExtractor – JDBC queries against the plaintext SQLite; validates
required tables before parsing
- SignalContact – Display name resolution priority:
system_joined_name → profile_joined_name →
profile_given_name+family → e164 → "Unknown"
- SignalChat – Thread model (individual and group)
- SignalMessage – Message model with MessageType enum
- ReportGenerator – Produces Signal-themed chat bubble HTML
## Signal schema compatibility
Targets Signal Android ≤5.15 (no aci column). Queries: recipient,
thread, message, groups, group_membership tables. Message type
classification uses the lower-5-bit mask from MessageTypes.kt:
0/2/3/4/5 → OUTGOING, 20 → INCOMING, 1 → CALL_OUTGOING,
21 → CALL_INCOMING, 22/25 → CALL_MISSED, else → SYSTEM.
## ExtraProperties alignment (vs WhatsApp/Threema parsers)
- USER_ACCOUNT_TYPE set to "message/x-signal-message" on every message
document (required for IPED's communication graph task).
- GROUP_ID uses the actual Signal group_id from recipient.group_id,
not a synthetic "SignalThread_<id>".
- MESSAGE_FROM / MESSAGE_TO / IS_GROUP_MESSAGE / PARTICIPANTS /
MESSAGE_DATE / DECODED_DATA / HASCHILD aligned with established
parser conventions.
## Category registration (CategoriesConfig.json)
- Chats → Signal: added application/x-signal-db and
application/x-signal-chat alongside existing UFED entry.
- Instant Messages Artifacts → Instant Messages: added
message/x-signal-message alongside x-threema-message,
x-whatsapp-message, etc.
## Known Phase-1 limitation
PARTICIPANTS does not include the device owner (self). Identifying self
requires matching against aci or SharedPreferences outside signal.db;
this is deferred to Phase 2.
## Tests (23 total)
Synthetic SQLite fixture without aci column mimics pre-v5.15 backups.
Coverage: named contacts, phone-only contacts, groups, sent/received/
null-body messages, emoji + HTML special chars, call types
(outgoing/incoming/missed), system message filtering, group outgoing
MESSAGE_TO (chat title, not "Unknown"), USER_ACCOUNT_TYPE on all
message docs, GROUP_ID equals actual Signal group_id, extractMessages
flag.
## Security: Template injection in ReportGenerator (evidence integrity)
Sequential String.replace() calls allowed user-controlled data containing
template placeholders (e.g. a group named "${messages}") to cascade through
substitution. A suspect could name a Signal group "${messages}" to cause the
forensic report to render all conversation messages in the title position
instead of the group name — corrupting evidence integrity.
Fix: replace all four placeholders in a single Matcher.replaceAll() pass.
Matcher.quoteReplacement() prevents replacement strings from being treated
as regex patterns. The replacement callback runs once per placeholder in
the original template; injected placeholders in substituted content are
never re-evaluated.
Adds testTemplateInjectionInGroupTitle to verify that a group named
"${messages}" renders the literal text in the title and does not cause
message content to be injected or duplicated.
## Also: Signal MIME types in CategoriesConfig.json
- Chats → Signal: application/x-signal-db and application/x-signal-chat
added alongside the existing UFED entry so forensic analysts find
Signal data under the expected category.
- Instant Messages Artifacts → Instant Messages: message/x-signal-message
added alongside message/x-threema-message, message/x-whatsapp-message, etc.
…s.xml Was incorrectly declared as a sub-class of itself. Correct parent type is message/x-chat-message, consistent with message/x-whatsapp-message, message/x-threema-message, and message/x-telegram-message.
Resolves the Phase-1 limitation where the device owner's identity was
absent from PARTICIPANTS and MESSAGE_FROM/MESSAGE_TO for outgoing messages.
Detection strategy: outgoing Signal messages carry from_recipient_id
pointing to the device owner's own recipient row. A single correlated
subquery selects the recipient who appears most frequently as
from_recipient_id in outgoing message types (type & 31 in {0,1,2,3,4,5}).
Returns null gracefully when no outgoing messages exist (Phase-1 fallback).
Behaviour changes:
- PARTICIPANTS: self is now the first entry in every chat, consistent
with the WhatsApp and Threema parsers. Group chats add self first,
then members from group_membership excluding self (no duplication).
Individual chats add self first, then the other party.
- MESSAGE_FROM: outgoing messages carry self's full identifier instead
of an empty string.
- MESSAGE_TO: incoming individual messages carry self's full identifier
instead of an empty string.
Tests (25 total, 0 failures):
- AbstractPkgTest: participants collector updated to capture ALL values
via metadata.getValues() — previously only the first entry per chat
was captured, which would have hidden self after Phase 2 reordering.
- testCallOutgoingIsFromMe: assertion updated from empty MESSAGE_FROM
to SELF_FULL_ID.
- testSelfInParticipants (new): verifies self appears once per chat
across all 3 chats.
- test_signal.db fixture: recipient 5 (Device Owner +5511999990005)
added; from_recipient_id on all outgoing messages set to 5; self
added to group_membership for the test group.
Author
|
Did another pass before asking for your time. A few things I fixed: The
|
- Re-throw SAXException before catch(Exception) so Tika's pipeline stop mechanism (WriteLimitReachedException) works correctly - Call shouldParseEmbedded before each chat report and message item, honouring IPED's per-item filtering logic (same fix applied to Android call log parser in a previous commit) - readResource() now logs a WARN when the HTML template or CSS file is missing from the classpath, instead of silently returning empty string that produces blank forensic reports - Fix misleading test comment: "→ 8 indexed" corrected to "→ 9 indexed (3+3+3)" — the assertion was always correct (EXPECTED_MESSAGE_DOCS = 9) All 25 Signal unit tests pass.
cleitonaugusto
force-pushed
the
#1602_signal_parser
branch
from
July 6, 2026 23:19
697a405 to
b9d6fd6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements a native Signal Messenger Android parser for IPED, extracting individual and group conversations from `signal.db` forensic acquisitions.
Architecture
Signal schema compatibility
Targets Signal Android pre-v5.15 (no `aci` column). Queries: `recipient`, `thread`, `message`, `groups`, `group_membership`. Message type classification uses the lower-5-bit mask from `MessageTypes.kt`.
ExtraProperties alignment
Security
Template substitution in `ReportGenerator` uses a single-pass `Matcher.replaceAll()` with `Matcher.quoteReplacement()` to prevent user-controlled group names containing `${placeholder}` strings from cascading through sequential `String.replace()` calls — which would corrupt the forensic HTML report.
Configuration
Device owner identification
`SignalExtractor.findSelfContact()` runs a single correlated subquery: it selects the `recipient` row whose `_id` appears most frequently as `from_recipient_id` in outgoing messages (`type & 31 IN (0,1,2,3,4,5)`). Returns `null` gracefully when no outgoing messages exist, preserving the Phase-1 fallback (empty string) without any exception.
Tests
25 tests via a synthetic SQLite fixture (`test_signal.db`) without `aci` column to prove pre-v5.15 compatibility. Coverage: named contacts, phone-only contacts, groups, sent/received/null-body messages, emoji + HTML special chars, call types (outgoing/incoming/missed), system message filtering, group outgoing `MESSAGE_TO`, `USER_ACCOUNT_TYPE` on all message docs, `GROUP_ID` equals actual Signal `group_id`, device owner in `PARTICIPANTS` for every chat, template injection prevention, `extractMessages` flag.
```
Tests run: 25, Failures: 0, Errors: 0, Skipped: 0
```
Closes #1602