Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for new absence operator (#33) #34

Merged
merged 1 commit into from
Jul 10, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,7 @@ _Note that not all of these are available in all versions of Ruby_
| &emsp;&emsp;_Lookbehind_ | `(?<=abc)` | &#x2713; |
| &emsp;&emsp;_Negative Lookbehind_ | `(?<!abc)` | &#x2713; |
| &emsp;&nbsp;_**Atomic**_ | `(?>abc)` | &#x2713; |
| &emsp;&nbsp;_**Absence**_ | `(?~abc)` | &#x2713; |
| &emsp;&nbsp;_**Back-references**_ | | &#x22f1; |
| &emsp;&emsp;_Named_ | `\k<name>` | &#x2713; |
| &emsp;&emsp;_Nest Level_ | `\k<n-1>` | &#x2713; |
Expand Down
1 change: 1 addition & 0 deletions lib/regexp_parser/expression/classes/group.rb
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ class Atomic < Group::Base; end
class Capture < Group::Base; end
class Passive < Group::Base; end
class Options < Group::Base; end
class Absence < Group::Base; end

class Named < Group::Capture
attr_reader :name
Expand Down
2 changes: 1 addition & 1 deletion lib/regexp_parser/lexer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# given syntax flavor.
module Regexp::Lexer

OPENING_TOKENS = [:capture, :options, :passive, :atomic, :named,
OPENING_TOKENS = [:capture, :options, :passive, :atomic, :named, :absence,
:lookahead, :nlookahead, :lookbehind, :nlookbehind
].freeze

Expand Down
2 changes: 2 additions & 0 deletions lib/regexp_parser/parser.rb
Original file line number Diff line number Diff line change
Expand Up @@ -470,6 +470,8 @@ def self.open_group(token)
exp = Group::Named.new(token)
when :capture
exp = Group::Capture.new(token)
when :absence
exp = Group::Absence.new(token)

when :lookahead
exp = Assertion::Lookahead.new(token)
Expand Down
5 changes: 4 additions & 1 deletion lib/regexp_parser/scanner/scanner.rl
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@

group_atomic = '?>';
group_passive = '?:';
group_absence = '?~';

assertion_lookahead = '?=';
assertion_nlookahead = '?!';
Expand All @@ -107,7 +108,7 @@
group_number_ref = group_ref . (('<' . group_number . group_level? '>') |
("'" . group_number . group_level? "'"));

group_type = group_atomic | group_passive | group_named;
group_type = group_atomic | group_passive | group_absence | group_named;


assertion_type = assertion_lookahead | assertion_nlookahead |
Expand Down Expand Up @@ -573,6 +574,7 @@
# Groups
# (?:subexp) passive (non-captured) group
# (?>subexp) atomic group, don't backtrack in subexp.
# (?~subexp) absence group, matches anything that is not subexp
# (?<name>subexp) named group
# (?'name'subexp) named group (single quoted version)
# (subexp) captured group
Expand All @@ -581,6 +583,7 @@
case text = text(data, ts, te).first
when '(?:'; emit(:group, :passive, text, ts, te)
when '(?>'; emit(:group, :atomic, text, ts, te)
when '(?~'; emit(:group, :absence, text, ts, te)

when /^\(\?<(\w*)>/
empty_name_error(:group, 'named group (ab)') if $1.empty?
Expand Down
2 changes: 2 additions & 0 deletions lib/regexp_parser/syntax/ruby/2.4.1.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ module Ruby
class V241 < Regexp::Syntax::Ruby::V240
def initialize
super

implements :group, Group::Absence
end
end

Expand Down
2 changes: 2 additions & 0 deletions lib/regexp_parser/syntax/tokens/group.rb
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ module Group
All = Group::Extended + Group::Named + Group::Atomic +
Group::Passive + Group::Comment

Absence = [:absence]

Type = :group
end

Expand Down
13 changes: 13 additions & 0 deletions test/parser/test_groups.rb
Original file line number Diff line number Diff line change
Expand Up @@ -106,4 +106,17 @@ def test_parse_comment
end
end

if RUBY_VERSION >= '2.4.1'
def test_parse_absence_group
t = RP.parse('a(?~b)c(?~d)e')

[1,3].each do |i|
assert t.expressions[i].is_a?(Group::Absence),
"Expected absence group, but got #{t.expressions[i].class.name}"

assert_equal :group, t.expressions[i].type
assert_equal :absence, t.expressions[i].token
end
end
end
end
7 changes: 7 additions & 0 deletions test/scanner/test_groups.rb
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,13 @@ class ScannerGroups < Test::Unit::TestCase
})
end

if RUBY_VERSION >= '2.4.1'
tests.merge!({
# New absence operator
'(?~abc)' => [0, :group, :absence, '(?~', 0, 3],
})
end

tests.each_with_index do |(pattern, (index, type, token, text, ts, te)), count|
define_method "test_scanner_#{type}_#{token}_#{count}" do
tokens = RS.scan(pattern)
Expand Down
6 changes: 3 additions & 3 deletions test/warnings.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
# Unused variable emitted by ragel
- "lib/regexp_parser/scanner.rb:1674: warning:
- "lib/regexp_parser/scanner.rb:1675: warning:
assigned but unused variable - testEof"

# Unavoidable duplicated character range tests
Expand All @@ -10,7 +10,7 @@
character class has duplicated range: /[a[b[^c]]]/"

# Warnings generated only under 1.9.3
- "lib/regexp_parser/scanner.rb:1675:
- "lib/regexp_parser/scanner.rb:1676:
warning: assigned but unused variable - _acts"
- "lib/regexp_parser/scanner.rb:1675:
- "lib/regexp_parser/scanner.rb:1676:
warning: assigned but unused variable - _nacts"