예제 #1
0
 def test_ngrams_for_tokens(self):
   self.assertEqual(text.ngrams_for_tokens(['one', 'two', 'three'], 1),
                    [('one',), ('two',), ('three',)])
   self.assertEqual(text.ngrams_for_tokens(['one', 'two', 'three'], 2),
                    [('one', 'two'), ('two', 'three')])
   self.assertEqual(text.ngrams_for_tokens(['one', 'two', 'three'], 3),
                    [('one', 'two', 'three')])
예제 #2
0
파일: mapper.py 프로젝트: wiseman/mefingram 
 def ngrams_for_text(self, s, n):
   tokens = text.title_tokens(s)
   return text.ngrams_for_tokens(tokens, n)