Skip to content

Commit 73812f7

Browse files
masatakemasatake
committed
main: add nulltag/z, a new extra
Close universal-ctags#4151. Consider a parser attempting to emit a null tag, a tag whose name is the empty string '\0'. Original Behavior: It warns "ignoring null tag..." if both parserDefinition::allowNullTag and tagEntryInfo::allowNullTag are unset. It does not warn if either parserDefinition::allowNullTag or tagEntryInfo::allowNullTag is set. It does not emit the null tag, even if allowNullTag is set. With This Change: The code now emits the null tag if: Either parserDefinition::allowNullTag or tagEntryInfo::allowNullTag is set, and The --extras=+0 (or --extras=+{nulltag}) option is specified. TODO: - versioning - updating the hacking guide - make readtags warn "unsupported" if it finds "!_TAG_FIELD_DESCRIPTION" Signed-off-by: Masatake YAMATO <[email protected]>
1 parent b28f59b commit 73812f7

File tree

18 files changed

+143
-2
lines changed

18 files changed

+143
-2
lines changed

‎Tmain/extras-long.d/stdout-expected.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ p pseudo no NONE no Include pseudo tags
77
q qualified no NONE no Include an extra class-qualified tag entry for each tag
88
r reference no NONE no Include reference tags
99
s subparser yes NONE no Include tags generated by subparsers
10+
z nulltag no NONE no Include tags with empty strings as their names
1011
- canonicalizedName yes Automake no Include canonicalized object name like libctags_a
1112
- linkName no Fortran no Linking name used in foreign languages
1213
- implicitClass no GDScript no Include tag for the implicitly defined unnamed class
@@ -25,6 +26,7 @@ p pseudo yes NONE no Include pseudo tags
2526
q qualified no NONE no Include an extra class-qualified tag entry for each tag
2627
r reference no NONE no Include reference tags
2728
s subparser yes NONE no Include tags generated by subparsers
29+
z nulltag no NONE no Include tags with empty strings as their names
2830
- canonicalizedName yes Automake no Include canonicalized object name like libctags_a
2931
- linkName no Fortran no Linking name used in foreign languages
3032
- implicitClass no GDScript no Include tag for the implicitly defined unnamed class
@@ -43,6 +45,7 @@ p pseudo yes NONE no Include pseudo tags
4345
q qualified no NONE no Include an extra class-qualified tag entry for each tag
4446
r reference no NONE no Include reference tags
4547
s subparser yes NONE no Include tags generated by subparsers
48+
z nulltag no NONE no Include tags with empty strings as their names
4649
- canonicalizedName yes Automake no Include canonicalized object name like libctags_a
4750
- linkName no Fortran no Linking name used in foreign languages
4851
- implicitClass no GDScript no Include tag for the implicitly defined unnamed class
@@ -61,6 +64,7 @@ p pseudo yes NONE no Include pseudo tags
6164
q qualified no NONE no Include an extra class-qualified tag entry for each tag
6265
r reference yes NONE no Include reference tags
6366
s subparser yes NONE no Include tags generated by subparsers
67+
z nulltag no NONE no Include tags with empty strings as their names
6468
- canonicalizedName yes Automake no Include canonicalized object name like libctags_a
6569
- linkName no Fortran no Linking name used in foreign languages
6670
- implicitClass no GDScript no Include tag for the implicitly defined unnamed class
@@ -79,6 +83,7 @@ p pseudo yes NONE no Include pseudo tags
7983
q qualified yes NONE no Include an extra class-qualified tag entry for each tag
8084
r reference yes NONE no Include reference tags
8185
s subparser yes NONE no Include tags generated by subparsers
86+
z nulltag no NONE no Include tags with empty strings as their names
8287
- canonicalizedName yes Automake no Include canonicalized object name like libctags_a
8388
- linkName no Fortran no Linking name used in foreign languages
8489
- implicitClass no GDScript no Include tag for the implicitly defined unnamed class

‎Tmain/json-output-format.d/stdout-expected.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@
2020
{"_type": "ptag", "name": "TAG_EXTRA_DESCRIPTION", "path": "fileScope", "pattern": "Include tags of file scope"}
2121
{"_type": "ptag", "name": "TAG_EXTRA_DESCRIPTION", "path": "guest", "pattern": "Include tags generated by guest parsers"}
2222
{"_type": "ptag", "name": "TAG_EXTRA_DESCRIPTION", "path": "inputFile", "pattern": "Include an entry for the base file name of every input file"}
23+
{"_type": "ptag", "name": "TAG_EXTRA_DESCRIPTION", "path": "nulltag", "pattern": "Include tags with empty strings as their names"}
2324
{"_type": "ptag", "name": "TAG_EXTRA_DESCRIPTION", "path": "pseudo", "pattern": "Include pseudo tags"}
2425
{"_type": "ptag", "name": "TAG_EXTRA_DESCRIPTION", "path": "qualified", "pattern": "Include an extra class-qualified tag entry for each tag"}
2526
{"_type": "ptag", "name": "TAG_EXTRA_DESCRIPTION", "path": "reference", "pattern": "Include reference tags"}

‎Tmain/kind-abnormal-spec.d/stdout-expected.txt

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,8 @@ r emit a tag with multi roles
1212
R emit a tag with multi roles(disabled by default) [off]
1313
f tag for testing field:
1414
n trigger notice output
15+
z emit a tag having an empty string
16+
Z don't emit a tag having an empty string
1517

1618
# list kinds-full
1719
#LETTER NAME ENABLED REFONLY NROLES MASTER DESCRIPTION
@@ -22,12 +24,14 @@ L ThisShouldNotBePrintedKindNameMustBeGiven yes no 0 NONE
2224
N nothingSpecial yes no 0 NONE emit a normal tag
2325
Q quit yes no 0 NONE stop the parsing
2426
R rolesDisabled no yes 2 NONE emit a tag with multi roles(disabled by default)
27+
Z dontEmitNullTag yes no 0 NONE don't emit a tag having an empty string
2528
b broken tag yes no 1 NONE name with unwanted characters
2629
d disabled no no 2 NONE a kind disabled by default
2730
e enabled yes no 2 NONE a kind enabled by default
2831
f fieldMaker yes no 0 NONE tag for testing field:
2932
n triggerNotice yes no 0 NONE trigger notice output
3033
r roles yes yes 4 NONE emit a tag with multi roles
34+
z emitNullTag yes no 0 NONE emit a tag having an empty string
3135

3236
# +K
3337
abnormal kindDefinition testing (no letter) input.x /^@$/;" no letter

‎Tmain/list-extras.d/stdout-expected.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ p pseudo yes NONE no Include pseudo tags
77
q qualified yes NONE no Include an extra class-qualified tag entry for each tag
88
r reference yes NONE no Include reference tags
99
s subparser yes NONE no Include tags generated by subparsers
10+
z nulltag yes NONE no Include tags with empty strings as their names
1011
- canonicalizedName yes Automake no Include canonicalized object name like libctags_a
1112
- linkName no Fortran no Linking name used in foreign languages
1213
- implicitClass no GDScript no Include tag for the implicitly defined unnamed class
@@ -25,6 +26,7 @@ p pseudo yes NONE no Include pseudo tags
2526
q qualified yes NONE no Include an extra class-qualified tag entry for each tag
2627
r reference yes NONE no Include reference tags
2728
s subparser yes NONE no Include tags generated by subparsers
29+
z nulltag yes NONE no Include tags with empty strings as their names
2830
- canonicalizedName yes Automake no Include canonicalized object name like libctags_a
2931
- linkName no Fortran no Linking name used in foreign languages
3032
- implicitClass no GDScript no Include tag for the implicitly defined unnamed class
@@ -43,3 +45,4 @@ p pseudo no NONE no Include pseudo tags
4345
q qualified no NONE no Include an extra class-qualified tag entry for each tag
4446
r reference no NONE no Include reference tags
4547
s subparser no NONE no Include tags generated by subparsers
48+
z nulltag no NONE no Include tags with empty strings as their names

‎Tmain/nulltag-extra.d/input.cst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Z
2+
z
3+

‎Tmain/nulltag-extra.d/run.sh

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Copyright: 2024 Masatake YAMATO
2+
# License: GPL-2
3+
4+
CTAGS=$1
5+
6+
. ../utils.sh
7+
8+
# The order of stdout and stderr lines is not stable.
9+
exit_if_win32 "$CTAGS"
10+
11+
# is_feature_available $CTAGS json
12+
13+
O="--options=NONE --language-force=CTagsSelfTest"
14+
15+
for fmt in u-ctags; do
16+
echo "# no extra ($fmt)"
17+
${CTAGS} $O -o - --output-format="$fmt" input.cst 2>&1
18+
19+
echo "# drop '0' extra ($fmt)"
20+
${CTAGS} $O -o - --output-format="$fmt" --extras=-z input.cst 2>&1
21+
22+
echo "# drop '{nulltag}' extra ($fmt)"
23+
${CTAGS} $O -o - --output-format="$fmt" --extras=-'{nulltag}' input.cst 2>&1
24+
25+
echo '# with --extras=+0 ($fmt)'
26+
${CTAGS} $O -o - --output-format="$fmt" --extras=+z input.cst 2>&1
27+
28+
echo "# with --extras=+{nulltag}' ($fmt)"
29+
${CTAGS} $O -o - --output-format="$fmt" --extras=+'{nulltag}' input.cst 2>&1
30+
done | sed -e 's/\.exe//'

‎Tmain/nulltag-extra.d/stderr-expected.txt

Whitespace-only changes.

‎Tmain/nulltag-extra.d/stdout-expected.txt

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# no extra (u-ctags)
2+
ctags: Notice: No options will be read from files or environment
3+
ctags: Notice: ignoring null tag in input.cst(line: 1, language: CTagsSelfTest)
4+
# drop '0' extra (u-ctags)
5+
ctags: Notice: No options will be read from files or environment
6+
ctags: Notice: ignoring null tag in input.cst(line: 1, language: CTagsSelfTest)
7+
# drop '{nulltag}' extra (u-ctags)
8+
ctags: Notice: No options will be read from files or environment
9+
ctags: Notice: ignoring null tag in input.cst(line: 1, language: CTagsSelfTest)
10+
# with --extras=+0 ($fmt)
11+
ctags: Notice: No options will be read from files or environment
12+
ctags: Notice: ignoring null tag in input.cst(line: 1, language: CTagsSelfTest)
13+
input.cst /^z$/;" z
14+
# with --extras=+{nulltag}' (u-ctags)
15+
ctags: Notice: No options will be read from files or environment
16+
ctags: Notice: ignoring null tag in input.cst(line: 1, language: CTagsSelfTest)
17+
input.cst /^z$/;" z

‎docs/man/ctags.1.rst

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1927,6 +1927,24 @@ The meaning of major extras is as follows (long-name flag/one-letter flag):
19271927

19281928
The etags mode enables the ``Unknown`` parser implicitly.
19291929

1930+
``nulltag``/``z``
1931+
Include tags (*null tags*) having empty strings as their names.
1932+
Generally speaking, trying to make a null tag is a sign of a parser bug
1933+
or broken input. ctags warns such trying or throws the
1934+
null tag away. To suppress the warnings, use ``--quiet`` option.
1935+
1936+
On the other hand null tags are valid in some languages.
1937+
Consider ``{"": val}`` in a JavaScript sourece code. The empty string is
1938+
valid as a key. If a parser intentionally makes a null tag (a valid null tag),
1939+
ctags doesn't warn but discard it by default.
1940+
1941+
The discards are due because some output formats may not consider null tags.
1942+
Though :ref:`tags(5) <tags(5)>`, the native format of Universal Ctags, doesn't forbid null tags,
1943+
it is questionable that client tools reading the tags output consider null tags.
1944+
1945+
With ``nulltag``/``z`` extra, you can force ctags to emit the nulltags. This extra
1946+
is effective only if the output format supports null tags.
1947+
19301948
``pseudo``/``p``
19311949
Include pseudo-tags. Enabled by default unless the tag file is
19321950
written to standard output. See :ref:`ctags-client-tools(7) <ctags-client-tools(7)>` about

‎main/entry.c

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1770,6 +1770,15 @@ static void writeTagEntry (tagEntryInfo *const tag)
17701770

17711771
DebugStatement ( debugEntry (tag); )
17721772

1773+
if (isTagExtraBitMarked(tag, XTAG_NULLTAG))
1774+
{
1775+
if (!writerCanPrintNullTag())
1776+
return;
1777+
1778+
if (!isXtagEnabled(XTAG_NULLTAG))
1779+
return;
1780+
}
1781+
17731782
#ifdef _WIN32
17741783
if (getFilenameSeparator(Option.useSlashAsFilenameSeparator) == FILENAME_SEP_USE_SLASH)
17751784
{
@@ -1939,10 +1948,17 @@ extern int makeTagEntry (tagEntryInfo *const tag)
19391948
if (tag->name [0] == '\0' && (!tag->placeholder))
19401949
{
19411950
if (! tag->allowNullTag)
1951+
{
19421952
error (NOTICE, "ignoring null tag in %s(line: %lu, language: %s)",
19431953
getInputFileName (), tag->lineNumber,
19441954
getLanguageName (tag->langType));
1945-
goto out;
1955+
goto out;
1956+
}
1957+
1958+
/* writeTagEntry decides whether ctags emits this tag or not.
1959+
* At this point, we just mark the tag as a null tag. */
1960+
if (! tag->placeholder)
1961+
markTagExtraBit(tag, XTAG_NULLTAG);
19461962
}
19471963

19481964
if (TagFile.cork)

‎main/parse.c

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5550,6 +5550,8 @@ typedef enum {
55505550
K_ROLES_DISABLED,
55515551
K_FIELD_TESTING,
55525552
K_TRIGGER_NOTICE,
5553+
K_EMIT_NULL_TAG,
5554+
K_DONT_EMIT_NULL_TAG,
55535555
KIND_COUNT
55545556
} CTST_Kind;
55555557

@@ -5631,6 +5633,8 @@ static kindDefinition CTST_Kinds[KIND_COUNT] = {
56315633
.referenceOnly = true, ATTACH_ROLES (CTST_RolesDisabledKindRoles)},
56325634
{true, 'f', "fieldMaker", "tag for testing field:" },
56335635
{true, 'n', "triggerNotice", "trigger notice output"},
5636+
{true, 'z', "emitNullTag", "emit a tag having an empty string"},
5637+
{true, 'Z', "dontEmitNullTag", "don't emit a tag having an empty string"},
56345638
};
56355639

56365640
typedef enum {
@@ -5820,6 +5824,16 @@ static void createCTSTTags (void)
58205824
case K_TRIGGER_NOTICE:
58215825
notice ("notice output for testing: %s", CTST_Kinds [i].name);
58225826
break;
5827+
case K_EMIT_NULL_TAG:
5828+
initTagEntry (&e, "", i);
5829+
e.allowNullTag = 1;
5830+
makeTagEntry (&e);
5831+
break;
5832+
case K_DONT_EMIT_NULL_TAG:
5833+
initTagEntry (&e, "", i);
5834+
e.allowNullTag = 0;
5835+
makeTagEntry (&e);
5836+
break;
58235837
}
58245838

58255839
if (quit)

‎main/writer-ctags.c

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ tagWriter uCtagsWriter = {
5555
.rescanFailedEntry = NULL,
5656
.treatFieldAsFixed = treatFieldAsFixed,
5757
.checkOptions = checkCtagsOptions,
58+
.canPrintNullTag = true,
5859
#ifdef _WIN32
5960
.overrideFilenameSeparator = overrideFilenameSeparator,
6061
#endif

‎main/writer-json.c

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -313,6 +313,7 @@ tagWriter jsonWriter = {
313313
.preWriteEntry = NULL,
314314
.postWriteEntry = NULL,
315315
.defaultFileName = "-",
316+
.canPrintNullTag = false,
316317
};
317318

318319
extern bool ptagMakeJsonOutputVersion (ptagDesc *desc, langType language CTAGS_ATTR_UNUSED,

‎main/writer.c

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,10 @@ extern bool writerCanPrintPtag (void)
117117
return (writer->writePtagEntry)? true: false;
118118
}
119119

120+
extern bool writerCanPrintNullTag (void)
121+
{
122+
return writer->canPrintNullTag;
123+
}
120124
extern bool writerDoesTreatFieldAsFixed (int fieldType)
121125
{
122126
if (writer->treatFieldAsFixed)

‎main/writer_p.h

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,8 @@ struct sTagWriter {
5454

5555
void (* checkOptions) (tagWriter *writer, bool fieldsWereReset);
5656

57+
bool canPrintNullTag;
58+
5759
#ifdef _WIN32
5860
enum filenameSepOp (* overrideFilenameSeparator) (enum filenameSepOp currentSetting);
5961
#endif /* _WIN32 */
@@ -95,6 +97,7 @@ extern bool ptagMakeCtagsOutputFilesep (ptagDesc *desc, langType language CTAGS_
9597
extern bool ptagMakeCtagsOutputExcmd (ptagDesc *desc, langType language CTAGS_ATTR_UNUSED, const void *data);
9698

9799
extern bool writerCanPrintPtag (void);
100+
extern bool writerCanPrintNullTag (void);
98101
extern bool writerDoesTreatFieldAsFixed (int fieldType);
99102

100103
extern void writerCheckOptions (bool fieldsWereReset);

‎main/xtag.c

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,8 @@ static xtagDefinition xtagDefinitions [] = {
7777
"Include tags generated by subparsers"},
7878
{ true, '\0', "anonymous",
7979
"Include tags for non-named objects like lambda"},
80+
{ false, 'z', "nulltag",
81+
"Include tags with empty strings as their names"},
8082
};
8183

8284
static unsigned int xtagObjectUsed;

‎main/xtag.h

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ typedef enum eXtagType { /* extra tag content control */
3434
XTAG_TAGS_GENERATED_BY_GUEST_PARSERS = XTAG_GUEST, /* Geany uses the old name */
3535
XTAG_SUBPARSER,
3636
XTAG_ANONYMOUS,
37+
XTAG_NULLTAG,
3738

3839
XTAG_COUNT
3940
} xtagType;
@@ -42,7 +43,7 @@ struct sXtagDefinition {
4243
bool enabled;
4344
/* letter, and ftype are initialized in the main part,
4445
not in a parser. */
45-
#define NUL_XTAG_LETTER '\0'
46+
#define NUL_XTAG_LETTER '\0' /* Nothing todo with NULLTAG. */
4647
unsigned char letter;
4748
const char* name; /* used in extra: field */
4849
const char* description; /* displayed in --list-extra output */

‎man/ctags.1.rst.in

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1927,6 +1927,24 @@ The meaning of major extras is as follows (long-name flag/one-letter flag):
19271927

19281928
The etags mode enables the ``Unknown`` parser implicitly.
19291929

1930+
``nulltag``/``z``
1931+
Include tags (*null tags*) having empty strings as their names.
1932+
Generally speaking, trying to make a null tag is a sign of a parser bug
1933+
or broken input. @CTAGS_NAME_EXECUTABLE@ warns such trying or throws the
1934+
null tag away. To suppress the warnings, use ``--quiet`` option.
1935+
1936+
On the other hand null tags are valid in some languages.
1937+
Consider ``{"": val}`` in a JavaScript sourece code. The empty string is
1938+
valid as a key. If a parser intentionally makes a null tag (a valid null tag),
1939+
@CTAGS_NAME_EXECUTABLE@ doesn't warn but discard it by default.
1940+
1941+
The discards are due because some output formats may not consider null tags.
1942+
Though tags(5), the native format of Universal Ctags, doesn't forbid null tags,
1943+
it is questionable that client tools reading the tags output consider null tags.
1944+
1945+
With ``nulltag``/``z`` extra, you can force ctags to emit the nulltags. This extra
1946+
is effective only if the output format supports null tags.
1947+
19301948
``pseudo``/``p``
19311949
Include pseudo-tags. Enabled by default unless the tag file is
19321950
written to standard output. See ctags-client-tools(7) about

0 commit comments

Comments
 (0)