Python Parser._tokenize Examples

Example #1

File: parser_test.py Project: klangner/bluenotepad

 def testTokenizerWithName(self):
     parser = Parser()
     tokens = parser._tokenize('count "Page loaded" events')
     self.assertEqual(3, len(tokens))
     self.assertEqual(("page loaded", "NAME"), tokens[1])

Example #2

File: parser_test.py Project: klangner/bluenotepad

 def testTokenizer(self):
     parser = Parser()
     tokens = parser._tokenize("count events")
     self.assertEqual(2, len(tokens))
     self.assertEqual([("count", ""), ("events", "")], tokens)