Python tokenize_primary Examples

Example #1

File: testexport.py Project: aymara/verbenet-editor

 def test_preposition(self):
     self.assertEqual(list(tokenize_primary("NP V de V-inf")), ["NP", "V", {"de"}, "V-inf"])

Example #2

File: testexport.py Project: aymara/verbenet-editor

 def test_qu(self):
     self.assertEqual(list(tokenize_primary("NP V Qu Psubj")), ["NP", "V", "Qu", "Psubj"])

Example #3

File: testexport.py Project: aymara/verbenet-editor

 def test_simple_split(self):
     self.assertEqual(list(tokenize_primary("NP V NP")), ["NP", "V", "NP"])