Class: YARD::Parser::Ruby::TokenResolver

Inherits:
Object
  • Object
show all
Includes:
Enumerable
Defined in:
lib/yard/parser/ruby/token_resolver.rb

Overview

Supports #each enumeration over a source’s tokens, yielding the token and a possible CodeObjects::Base associated with the constant or identifier token.

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(source, namespace = Registry.root) ⇒ TokenResolver

Creates a token resolver for given source.

Parameters:

  • source (String) —

    the source code to tokenize

  • namespace (CodeObjects::Base) (defaults to: Registry.root) —

    the object/namespace to resolve from

Raises:



15
16
17
18
19
 
# File 'lib/yard/parser/ruby/token_resolver.rb', line 15

def initialize(source, namespace = Registry.root)
  @tokens = RubyParser.parse(source, '(tokenize)').tokens
  raise ParserSyntaxError if @tokens.empty? && !source.empty?
  @default_namespace = namespace
end

Class Method Details

.state_attr(*attrs) ⇒ Object 



91
92
93
94
95
96
97
 
# File 'lib/yard/parser/ruby/token_resolver.rb', line 91

def self.state_attr(*attrs)
  attrs.each do |attr|
    define_method(attr) { @states.last[attr.to_sym] }
    define_method("#{attr}=") {|v| @states.last[attr.to_sym] = v }
    protected attr, :"#{attr}="
  end
end

Instance Method Details

#each {|token, object| ... } ⇒ Object

Iterates over each token, yielding the token and a possible code object that is associated with the token.

Examples:

Yielding code objects

r = TokenResolver.new("A::B::C")
r.each do |tok, obj|
  if obj
    puts "#{tok[0]} -> #{obj.path.inspect}"
  else
    puts "No object: #{tok.inspect}"
  end
end

# Prints:
# :const -> "A"
# No object: [:op, "::"]
# :const -> "A::B"
# No object: [:op, "::"]
# :const -> "A::B::C"

Yield Parameters:

  • token (Array(Symbol,String,Array(Integer,Integer))) —

    the current token object being iterated

  • object (CodeObjects::Base, nil) —

    the fully qualified code object associated with the current token, or nil if there is no object for the yielded token.



45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
 
# File 'lib/yard/parser/ruby/token_resolver.rb', line 45

def each
  @states = []
  push_state
  @tokens.each do |token|
    yield_obj = false

    if skip_group && [:const, :ident, :op, :period].include?(token[0])
      yield token, nil
      next
    else
      self.skip_group = false
    end

    case token[0]
    when :const
      lookup(token[0], token[1])
      yield_obj = true
      self.last_sep = nil
    when :ident
      lookup(token[0], token[1])
      yield_obj = true
      self.last_sep = nil
    when :op, :period
      self.last_sep = token[1]
      unless CodeObjects.types_for_separator(token[1])
        self.object = nil
        self.last_sep = nil
      end
    when :lparen
      push_state
    when :rparen
      pop_state
    else
      self.object = nil
    end

    yield token, (yield_obj ? object : nil)

    if next_object
      self.object = next_object
      self.next_object = nil
    end
    self.skip_group = true if yield_obj && object.nil?
  end
end