TokenizeOptions

Stentor API Reference

---

Stentor API Reference / stentor-utils/src / TokenizeOptions

Interface: TokenizeOptions

Defined in: packages/stentor-utils/src/string/tokenize.ts:3

Properties

removeStopWords?

optional removeStopWords: boolean

Defined in: packages/stentor-utils/src/string/tokenize.ts:8

If true, removes common stop words from the tokenized output. Default is false.