Python NGramTransformer.n_gram Examples

Example #1

File: test_transform.py Project: shannonyu/classipy

 def test_trigram(self):
     expected = TestNGramTransformer.tokensA + ['a+b', 'b+c'] + [
         'a+b+c'
     ] + TestNGramTransformer.tokensB + ['A+B', 'B+C'] + ['A+B+C']
     t = NGramTransformer(Sentinel(), n=3)
     self.assertListEqual(expected,
                          list(t.n_gram(TestNGramTransformer.segments)))

Example #2

File: test_transform.py Project: conerade67/classipy

 def test_bigram(self):
     expected = TestNGramTransformer.tokensA + [
         'a+b', 'b+c'
     ] + TestNGramTransformer.tokensB + [
         'A+B', 'B+C'
     ]
     t = NGramTransformer(Sentinel(), n=2)
     self.assertListEqual(expected,
                          list(t.n_gram(TestNGramTransformer.segments)))

Example #3

File: test_transform.py Project: conerade67/classipy

 def test_unigram(self):
     expected = TestNGramTransformer.tokensA + TestNGramTransformer.tokensB
     t = NGramTransformer(Sentinel(), n=1)
     self.assertListEqual(expected,
                          list(t.n_gram(TestNGramTransformer.segments)))

Example #4

File: test_transform.py Project: shannonyu/classipy

 def test_unigram(self):
     expected = TestNGramTransformer.tokensA + TestNGramTransformer.tokensB
     t = NGramTransformer(Sentinel(), n=1)
     self.assertListEqual(expected,
                          list(t.n_gram(TestNGramTransformer.segments)))