Date Time Parsing

[techpranav]

Below is grammar for matching date time.
I am trying to parse date time in different formats
02.03.2022 00.55.23 AM
02/03/2022 00.55.23 PM
02-03-2022

But, it's not working as expected.

How can I fix this issue?

`grammar Test;

options {
caseInsensitive = true;
}

stmt1 : datetime_expr;
//stmt : PRINT TYPE any_string SELECT fields WHERE query_expr';';
any_string : arg_string | QUOTE (arg_string)* QUOTE;

arg_string : (SINGLE_WORD | keywords)+;

fields: (field)+ ;

field : (any_string ( LEFT_BRKT any_string RIGHT_BRKT)? | subfield );

subfield : (DOT field)+ ;

query_expr : query_expr ops query_expr
| any_string
;

//expr : query_expr comparators query_expr
// ;

comparators : (EQ | LT_EQ | LT |GT | GT_EQ );
ops : AND | OR ;
keywords : PRINT | TYPE | PROPERTY | SELECT | DOT;
QUOTE : ''' | '"' ;
TYPE : 'type';
PRINT : 'print';
SELECT : 'select';
WHERE : 'where';
PROPERTY : 'property';
AND : '&&';
OR : '||';
EQ : '==';
LT : '<';
LT_EQ : '<=';
GT_EQ : '>=';
GT : '>';

DOT : '.';
//SYMBOL : ~["' ] ;
//ALPHA : [\p{L}\p{Nl}]+;
//DIGIT : [\p{Nd}]+;

BAD : . -> skip;
WS : [ \t\r\n]+ -> skip ;

datetime_expr : iso_datetime | us_datetime | uk_datetime | custom_datetime ;

iso_datetime : iso_date 'T' iso_time ;
us_datetime : us_date ' ' us_time ;
uk_datetime : uk_date ' ' uk_time ;
custom_datetime : date_time | time_date | date_time_millis | time_date_millis ;

iso_date : YEAR '-' MONTH '-' DAY ;
uk_date : DAY DOT MONTH DOT YEAR ;
us_date : MONTH '/' DAY '/' YEAR ;
date_time : DAY_MONTH_YEAR ' ' HOURS ':' MINUTES ':' SECONDS ;
time_date : HOURS ':' MINUTES ':' SECONDS ' ' DAY_MONTH_YEAR ;
date_time_millis : DAY_MONTH_YEAR ' ' HOURS ':' MINUTES ':' SECONDS DOT MILLISECONDS ;
time_date_millis : HOURS ':' MINUTES ':' SECONDS DOT MILLISECONDS ' ' DAY_MONTH_YEAR ;

iso_time : HOURS ':' MINUTES ':' SECONDS ;
us_time : HOURS ':' MINUTES ':' SECONDS AP 'M' ;

AP : [AP] ;
uk_time : HOURS '.' MINUTES ;

DAY_MONTH_YEAR : DAY MONTH YEAR ;
YEAR_MONTH_DAY : YEAR MONTH DAY ;

YEAR : [1-9][0-9][0-9][0-9] ;
MILLISECONDS : [0-9]{3} ;
DAY : '0'[1-9] | [12][0-9] | '3'[01] ;
MONTH : '0'[1-9] | '1'[0-2] ;
MINUTES : [0-5][0-9] ;
HOURS : '0'[0-9] | '1'[0-9] | '2'[0-3] ;
SECONDS : [0-5][0-9] ;

SINGLE_WORD: [A-Za-z0-9:!@#$%^&_\/]+;
//SINGLE_WORD_DOT: [A-Za-z0-9:!@#$%^&_\/.-]+;
`

[ericvergnaud]

antlr recognizes tokens before it tries to combine them into grammar rules so ’03’ will always be seen as a DAY, because DAY is defined before MONTH in your grammar you should convert your datetime grammar rules to lexer rules, and declare YEAR/MONTH/DAY/… as lexer fragments

…

Le 23 mars 2023 à 17:13, techpranav ***@***.***> a écrit : Below is grammar for matching date time. I am trying to parse date time in different formats 02.03.2022 00.55.23 AM 02/03/2022 00.55.23 PM 02-03-2022 But, it's not working as expected. <https://user-images.githubusercontent.com/30039186/227266222-17f1d5d9-d823-4404-9569-aee26a467d12.png> How can I fix this issue? `grammar Test; options { caseInsensitive = true; } stmt1 : datetime_expr; //stmt : PRINT TYPE any_string SELECT fields WHERE query_expr';'; any_string : arg_string | QUOTE (arg_string)* QUOTE; arg_string : (SINGLE_WORD | keywords)+; fields: (field)+ ; field : (any_string ( LEFT_BRKT any_string RIGHT_BRKT)? | subfield ); subfield : (DOT field)+ ; query_expr : query_expr ops query_expr | any_string ; //expr : query_expr comparators query_expr // ; comparators : (EQ | LT_EQ | LT |GT | GT_EQ ); ops : AND | OR ; keywords : PRINT | TYPE | PROPERTY | SELECT | DOT; QUOTE : ''' | '"' ; TYPE : 'type'; PRINT : 'print'; SELECT : 'select'; WHERE : 'where'; PROPERTY : 'property'; AND : '&&'; OR : '||'; EQ : '=='; LT : '<'; LT_EQ : '<='; GT_EQ : '>='; GT : '>'; DOT : '.'; //SYMBOL : ~["' ] ; //ALPHA : [\p{L}\p{Nl}]+; //DIGIT : [\p{Nd}]+; BAD : . -> skip; WS : [ \t\r\n]+ -> skip ; datetime_expr : iso_datetime | us_datetime | uk_datetime | custom_datetime ; iso_datetime : iso_date 'T' iso_time ; us_datetime : us_date ' ' us_time ; uk_datetime : uk_date ' ' uk_time ; custom_datetime : date_time | time_date | date_time_millis | time_date_millis ; iso_date : YEAR '-' MONTH '-' DAY ; uk_date : DAY DOT MONTH DOT YEAR ; us_date : MONTH '/' DAY '/' YEAR ; date_time : DAY_MONTH_YEAR ' ' HOURS ':' MINUTES ':' SECONDS ; time_date : HOURS ':' MINUTES ':' SECONDS ' ' DAY_MONTH_YEAR ; date_time_millis : DAY_MONTH_YEAR ' ' HOURS ':' MINUTES ':' SECONDS DOT MILLISECONDS ; time_date_millis : HOURS ':' MINUTES ':' SECONDS DOT MILLISECONDS ' ' DAY_MONTH_YEAR ; iso_time : HOURS ':' MINUTES ':' SECONDS ; us_time : HOURS ':' MINUTES ':' SECONDS AP 'M' ; AP : [AP] ; uk_time : HOURS '.' MINUTES ; DAY_MONTH_YEAR : DAY MONTH YEAR ; YEAR_MONTH_DAY : YEAR MONTH DAY ; YEAR : [1-9][0-9][0-9][0-9] ; MILLISECONDS : [0-9]{3} ; DAY : '0'[1-9] | [12][0-9] | '3'[01] ; MONTH : '0'[1-9] | '1'[0-2] ; MINUTES : [0-5][0-9] ; HOURS : '0'[0-9] | '1'[0-9] | '2'[0-3] ; SECONDS : [0-5][0-9] ; SINGLE_WORD: [A-Za-z0-9:!@#$%^&_\/]+; //SINGLE_WORD_DOT: [A-Za-z0-9:!@#$%^&_\/.-]+; ` — Reply to this email directly, view it on GitHub <#4205>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZNQJFBRBLQ46YDSMXR3MTW5RZB5ANCNFSM6AAAAAAWFNOZNI>. You are receiving this because you are subscribed to this thread.

[techpranav]

Thank you for the response.
I tried changing date time grammar rules to lexer rules and declared fragments, but still I could not get the expected result.
Can you give me a sample code, I tried multiple ways but no luck.

[jimidle]

If all you are trying to do is parse dates, then ANTLR isn't really the right tool. If you are parsing as part of a bigger grammar, then it is probably still easier to consume the date string as one token and pass it off to some other library that already does it. Which language are you targeting?

As your grammar stands it is impossible for it to work. ANTLR grammars are not something that you can read about for 15 minutes then give it a try I am afraid. I don't wish to be mean or insulting here, but you are not going to succeed with this as a starting point. First read the book or online tutorials and then look at some example grammars for inspiration.

Here is some explanations that may help you move forward:

You are basically trying to define your problem as a syntax directed lexing system where the parser runs and then asks the lexer if the next token can be X. ANTLR does not do that. It first converts your text into the set of tokens, then the parse rules run on those tokens. The tokens don't change according to the parser rule. So, all your lexer tokens are ambiguous between themselves, AND your DAY_MONTH_YEAR etc are ambiguous again. ANTLR does not "know" what you mean here, it will just match the first LEXER token that is longest in order of the lexer rules as you list them top to bottom.

So, if I have two lexer rules:

JIM1 : 'JIM' ;
JIM2 : 'JIM' ;

The lexer will never return JIM2 - the parser has no influence on what tokens the lexer will generate. So I need to use one token:

JIM: 'JIM' ;

And use it in the parser where ever that is a valid input.

So you rules with numbers are all incorrect. You need something like this:

time: NUMBER ':' NUMBER ':' NUMBER ( AM | PM) ;

NUMBER: [0-9]+ ;
AM: 'AM' ;
PM: 'PM';

But even then, are you sure that AM and PM won't clash somewhere else, such as if your grammar has arbitrary variable names?

Also your parser rules are completely ambiguous, which ANTLR might be able to sort out if your lexer rules could work, but will be very inefficient.

Basically you are pursuing an X Y problem. You need to step away from this solution and first tell us what you are trying to achieve. We know you want to parse some dates from some source that is some kind of query, but where are these coming from? Where will this date parser run? What is the output? What are you trying to achieve?

[KvanTTT]

So, if I have two lexer rules:

JIM1 : 'JIM' ;
JIM2 : 'JIM' ;

The lexer will never return JIM2 - the parser has no influence on what tokens the lexer will generate

BTW, ANTLR reports TOKEN_UNREACHABLE warning for such rules.